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Abstract 

The  Air  Force  and  the  rest  of  the  Department  of  Defense  rely  on  valid  information 
to  make  National  Security  decisions.  The  veracity  of  the  information  used  to  make  those 
decisions  can  dramatically  affect  which  course  of  action  our  military  will  take. 

Therefore,  it  is  important  that  our  leaders  be  able  to  recognize  if  they  are  being  deceived. 
This  study  examines  the  results  of  training  five  categories  of  deception  cues  to  190  Air 
Force  Officers.  The  officers  were  tested  to  determine  their  baseline  deception  detection 
abilities,  then  trained  on  the  deception  cues.  Arousal,  Emotion,  Cognitive  Effort, 
Communicator  Tactics,  and  Memory  Processes,  then  retested  to  determine  how  much 
information  they  retained.  The  results  of  this  study  show  that  there  is  a  return  on  the 
investment  in  training  deception  detection  for  at  least  four  of  the  deception  cues. 
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DETERMINATION  OF  THE  TRAINABILITY 
OF  DECEPTION  DETECTION  CUES 


I.  Introduction 


Background 

The  word  “deception”  commonly  generates  negative  connotations,  and  in  some 
instances,  rightfully  so.  Nobody  wants  to  be  deceived.  Yet,  while  the  very  thought  of 
being  deceived  stirs  animosity  in  most  individuals,  those  same  individuals  are  probably 
guilty  of  using  deception  on  more  than  one  occasion.  Hence,  the  use  of  deception  may  be 
more  ubiquitous  than  commonly  known.  Therefore,  being  able  to  identify  when 
deception  is  being  employed  would  be  an  advantage  for  the  common  person. 

Deception  is  used  in  everyday  life  by  most  people  in  events  as  simple  as  buying  items 
at  a  swap  meet  or  garage  sale  or  during  the  purchase  of  a  car.  The  buyer,  in  an  attempt  to 
pay  the  least  possible,  will  intentionally  hide  how  much  he  is  willing  to  pay;  the  seller,  in 
an  attempt  to  get  the  most  for  her  object,  will  intentionally  hide  the  least  she  is  willing  to 
take.  Even  in  playing  “Go-Fish”  with  our  children  or  poker  with  our  friends,  we  employ 
deception;  we  do  not  want  the  other  person  knowing  what  is  in  our  hands. 

Some  would  argue  that  the  above  examples  do  not  constitute  deception  because  they 
are  expected  practices  when  playing  cards  or  purchasing  certain  items.  In  Information 
Manipulation  Theory,  McCornack  (1992)  discusses  Grice’s  (1989)  four  maxims  of 
communication  which  relate  to  these  situations.  Grice  (1989)  explains  quality,  quantity. 
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relation,  and  manner  and  how  they  pertain  to  eertain  eommunieation  eategories 
(MeCornaek,  1992:5;  Griee,  1989:26-27).  Quantity  refers  to  the  amount  of  information 
provided;  as  mueh  as  is  neeessary  and  no  more.  Quality  refers  to  the  veraeity  of  the 
eommunieation;  the  level  to  whieh  partieipants  are  antieipated  to  eonvey  truthfulness. 

The  relation  maxim  refers  to  partieipants  making  their  eommunieation  eontribution 
relevant  to  the  situation  at  hand  given  established  eonstraints.  Manner  refers  to  how  what 
is  said  viee  what  is  said  with  the  expeetation  that  partieipants  will  avoid  obseurity  and 
ambiguity.  While  the  above  examples  are  generally  aeeeptable  forms  of  deeeption  that 
fall  within  the  realm  of  Griee’s  (1989)  four  maxims,  this  researeh  will  eoneentrate  on  the 
less  aeeeptable  forms  of  deeeption  in  whieh  the  offended  party  is  expeeting  truthfulness 
and  is  harmed  in  some  way.  More  speeifieally  this  researeh  is  eoneemed  with  teaehing 
our  military  members  how  to  reeognize  eues  that  are  displayed  when  deeeption  is  being 
employed. 

Deception  in  the  Military 

Military  strategists  have  long  used  deeeptive  teehniques  to  aehieve  a  taetieal 
advantage.  A  perfeet  example  of  this  strategy  is  how  Japan  led  the  United  States  (US) 
into  believing  that  Japan  was  still  negotiating  restoration  of  trade  embargos,  oil  supplies 
from  the  Duteh  East  Indies,  and  peaee  negotiations  with  China  while  the  Japanese  Fleet 
was  bearing  down  on  Pearl  Harbor  (Friedrieh,  1991:39-41).  The  US  leaders  were 
suspieious  of  Japan’s  intentions  and  knew  that  they  were  preparing  for  war,  but  thought 
as  long  as  negotiations  were  eontinuing,  no  hostile  aetion  would  take  plaee.  The 
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Japanese  were  sueeessful  in  lulling  the  US  into  a  less  eautious  state  of  mind  than  the  US 
would  have  been  had  negotiations  not  been  taking  plaee. 

The  US  military  suceessfully  used  deeeption  during  Desert  Storm  by  eausing  the  Iraqi 
leadership  to  believe  they  were  going  to  be  attaeked  direetly  from  Kuwait.  General 
Norman  Sehwarzkopf  sent  a  major  attaek  body  around  to  the  west  and  north  with  his 
famous  “Hail  Mary”  move  (Sehwarzkopf,  1992:362,  408;  Camor,  2002:426).  This 
deeeption  suecessfully  trieked  the  Iraqi’s  into  eoneentrating  their  defenses  in  the  south. 
Although  there  is  little  doubt  the  US  would  have  won  a  direet  assault,  this  deeeption  may 
have  saved  numerous  Ameriean  lives. 

In  the  proeess  of  deeeiving,  most  people  uneonsciously  and  indeliberately  transmit 
evidenee  of  their  intent  to  deeeive  in  the  form  of  ones  (Buller,  1994:209).  One  would 
think  that  warning  an  individual  of  the  potential  for  deeeption  would  inerease  vigilanee 
and  ability  to  detect  deception;  however,  research  has  not  supported  this  claim.  Primed 
interviewers  were  no  better  at  detecting  deception  than  the  unprimed  (Toris  and  DePaulo, 
1984:1063-1073).  However,  the  primed  interviewers  were  more  suspicious  of  even  the 
honest  interviewees.  The  limitation  to  this  study,  according  to  the  interpretation  of  this 
literature  review,  is  that  the  “primed”  interviewers  were  not  trained  at  deception 
detection,  merely  informed  to  be  aware  of  deception. 

Problem 

This  research  evaluated  the  effectiveness  of  teaching  deception  cues  to  study 
participants.  There  were  five  categories  of  cues  evaluated:  arousal,  emotion. 
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communicator  tactics,  cognitive  effort,  and  memory  process.  Although  most  people 
believe  they  are  good  at  deteeting  deeeption,  they  are  actually  only  correct  slightly  more 
than  half  the  time  (Cao,  et  al(b),  2003:1;  Feeley  &  Young,  2000:105;  Stiff  et  al, 

1992:326;  Zuekerman,  et  al,  1984:519).  One  eould  make  almost  as  accurate  a 
determination  as  to  whether  he  was  being  deeeived  or  not  by  a  simple  coin  toss. 
Therefore,  for  national  seeurity  reasons,  it  is  in  the  best  interest  of  the  military  serviees  to 
teach  their  members  how  to  recognize  deception  cues. 

Military  members,  leaders  especially,  must  be  able  to  reeognized  deeeption  cues.  In  a 
meeting  between  Great  Britain’s  prime  minister,  Neville  Chamberlain,  and  Adolf  Hitler, 
Hitler’s  ability  to  deeeive  gave  him  the  upper  hand.  While  Hitler  was  mobilizing  his 
troops  in  preparation  for  attaeking  the  Czechs,  he  needed  to  eonvinee  Chamberlain  that 
peace  was  possible  between  Germany  and  Czeehoslovakia.  Hitler  was  stalling  for  time 
and  needed  Chamberlain  to  convince  the  Czechs  that  peace  was  possible  to  keep  the 
Czechs  from  building  their  forees.  Hitler’s  deceit  was  successful  and  Chamberlain 
reported  to  Parliament  that  Hitler  “means  what  he  says.”  (Ekman,  1985:15-16)  Although 
rudimentary  by  today’s  standards,  this  is  a  good  example  of  the  need  for  us  to  reeognize 
deeeption  eues.  Had  Chamberlain  reeognized  that  Hitler  was  lying,  history  may  be 
different  today. 

Previous  research  has  resulted  in  development  of  a  eomputer-based  training 
(CBT)  system  to  train  deception  eues.  In  a  2002  study,  the  effeetiveness  of  the  CBT 
was  compared  to  that  of  the  instructor  lecture  method  for  teaehing  individuals  how  to 
recognize  cues  that  indicate  deception  was  being  employed  (Lankowski,  2003; 

Dzubinski,  2003;  Knode,  2003;  Roekwell;  2003).  It  was  discovered  that  the  CBT  was 
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at  least  as  good  as,  but  not  significantly  more  effective  than  that  of  instructor  lecture. 

The  CBT  has  since,  however,  been  improved  with  a  more  interactive  option  that 
permits  the  student  to  conduct  keyword  searches,  ask  the  software  questions,  and 
navigate  through  the  lecture  curriculum  to  either  repeat  portions  or  skip  portions  the 
student  feels  he  is  familiar  with. 

The  Research  Question 

By  learning  to  recognize  Burgeon’s  (in  press)  deception  cues,  one  should  improve 
their  accuracy  in  detecting  deception.  According  to  previous  research,  training  deception 
detection  cues  via  CBT  proves  to  be  at  least  as  and  possibly  slightly  more  effective  than 
live  instruction  (Lankowski,  2003;  Dzubinski,  2003;  Knode,  2003;  Rockwell;  2003).  The 
above  two  assertions  leads  to  the  following  research  question:  Which  deception  cues, 
amongst  Burgoon’s  (in  press)  five  categories  of  cues,  are  effectively  trained  via  CBT? 

The  rest  of  this  research  paper  examines  the  research  question  above  in  great  detail. 
Chapter  two  of  this  paper  involves  an  extensive  literature  review  to  ascertain  what  is 
already  known  of  the  subject  of  deception  detection,  the  use  of  CBT,  and  the  use  of  third- 
party  observations  in  training  scenarios.  Chapter  three  discusses  the  research 
methodology  used  to  determine  which  deception  cues  are  best  trained  using  the  CBT. 
Chapter  four  explains  the  results  of  the  data  obtained  during  the  research  experiment. 
Chapter  five  is  an  in-depth  analysis,  explains  the  implications  of  the  analysis,  and 
discusses  the  scope  and  limitations  of  the  use  of  the  new  knowledge  discovered  as  a 
result  of  this  study. 
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II,  Literature  Review 


Overview 

This  chapter  reviews  and  summarizes  much  of  what  is  already  known  about  deception 
detection.  Studies  conducted  in  the  area  of  deception  date  back  to  1920  (Burgoon, 
1994:208).  Burgoon  referenced  Marston  (1920)  regarding  changes  in  vocalic  patterns 
during  deception.  Although  the  relevance  of  earlier  studies  is  clearly  important  to 
continued  work  in  the  field,  there  was  an  explosion  in  interest  in  deception  detection  with 
a  subsequent  increase  in  the  number  of  studies  during  the  1980s  and  1990s  (Buller  and 
Burgoon,  1994a;  Ekman,1985;  Burgoon,  1990;  Zuckerman,  Koestner,  and  Alton,  1984; 
Meyer,  1992;  Burgoon  et  al,  1992) 

This  research  examines  five  distinct  yet  closely  aligned  categories  of  cues  used  in 
deception  detection:  Arousal,  Emotion,  Communicator  Tactics,  Memory  Process,  and 
Cognitive  Effort.  The  researeh  will  all  be  conducted  via  CBT.  This  research  also  uses  a 
third  party  for  determining  whether  or  not  deception  has  oecurred.  Therefore,  this 
literature  review  also  examines  previous  research  conducted  in  the  use  of  CBT  and  third- 
party  observations. 

Defining  Deception 

To  begin  a  study  of  deception  detection,  the  definition  of  deception  must  be 
examined.  Definitions  of  deception  detection  cover  a  range  of  behaviors.  Some  of  the 
generally  accepted  meanings  of  deception  are: 

•  The  aet  of  deceiving,  cheating,  hoodwinking,  misleading,  or  deluding 
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•  The  fact  or  condition  of  being  deceived,  fooled,  or  deluded 

•  A  characteristic,  arrangement,  or  situation  that  deceives  or  deludes  with  or 
without  calculated  intent  (Webster,  1961) 

In  addition  to  the  dictionary  definition  of  deception,  most  scholars  who  study 
deception  posit  their  own  definition.  Two  scholars  who  have  done  extensive  research  in 
the  area  of  deception  detection,  Buller  and  Burgoon  (1994a:  192),  define  deception  as  “the 
intent  to  deceive  a  target  by  controlling  information  (e.g.,  transmitting  verbal  and 
nonverbal  messages,  and/or  manipulating  situational  cues)  to  alter  the  target’s  beliefs  or 
understanding  in  a  way  that  the  deceiver  knows  is  false.” 

Ekman  (1985:28)  has  a  simpler  definition  of  deception.  He  defines  a  liar  and 
deceiver  the  same:  a  person  that  intends  to  mislead  another.  He  continues  that  deceit  is  a 
deliberate  act  and  codifies  deceit  into  two  categories:  falsification  and  concealment. 
Concealment  is  passive  deceit,  requiring  no  action,  no  construction  of  a  story  or  lie. 
Falsification,  on  the  other  hand,  is  active  deceit.  It  requires  the  deceiver  to  construct  a  lie 
and  attempt  to  persuade  the  receiver  that  the  deceiver’s  rendition  is  the  truth. 

Concealment  often  appears  to  be  the  preferred  method  of  deceit  because  it 

•  Lessens  the  impact  on  the  deceiver  should  the  truth  be  discovered 

•  Is  considered  less  reprehensible  than  falsification 

•  Requires  no  memorization,  as  falsification  does,  to  maintain  consistency 

Further  research  from  Burgoon  and  Buller  (1994: 155)  defines  deception  as  “...a 
deliberate  act  perpetrated  by  a  sender  to  engender  in  a  receiver  beliefs  contrary  to  what 
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the  sender  believes  is  true  to  put  the  receiver  at  a  disadvantage.”  For  the  purposes  of  this 
study,  the  relatively  simple  definition  of  deception  is  that  of  knowingly  conveying  an 
untruth  to  another. 

Deception  Cues 

Now  that  the  definition  of  deception  has  been  explored,  the  next  step  is  to  examine 
what  cues  reveal  the  possibility  that  deception  is  being  employed.  A  deceiver’s  behavior 
can  inadvertently  reveal  his  deceit.  (Buller  &  Burgoon,  1994:209)  People  deceive  for 
multiple  reasons.  One  could  tell  a  “white  lie”  to  spare  someone’s  feelings  (i.e.,  “I  love 
your  new  hairdo”)  or  tell  a  complete  fabrication  to  save  his  life  (i.e.,  telling  a  police 
detective,  “I  didn’t  kill  the  man.”)  The  intent  of  the  lie,  the  level  of  deception  involved, 
and  the  consequences  of  being  discovered  invoke  varying  emotional  states  that  may 
reveal  themselves  through  the  individual’s  behavior  (Burgoon,  1990:357-365).  It  must 
be  clearly  understood  that  many  of  the  cues  or  behaviors  being  observed  are  brought  on 
by  emotional  responses  (Buller  &  Burgoon,  1994:  209).  These  emotional  responses 
could  be  due  to  the  individual  being  deceptive,  or  those  responses  could  be  for  a  variety 
of  other  reasons  that  invoke  emotional  responses.  The  sender  could  be  upset,  nervous,  or 
excited  due  to  a  situation  completely  removed  from  the  communication  at  hand.  Hence, 
the  observation  of  the  cues  discussed  later  does  not  necessarily  indicate  deception.  In 
fact,  a  truth  teller  will  begin  to  display  these  cues  when  he  perceives  that  the  receiver 
believes  he  is  lying.  If  the  receiver  then  falsely  interprets  the  new  behavior,  he  will  only 
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become  more  suspicious.  This  incorrect  interpretation  is  referred  to  as  the  “Othello 
Error”  (Ekman,  1985:132). 

Ekman  (1985:39)  defines  deception  clues  as  indicators  that  the  deceiver  is  not  being 
truthful,  and  leakage  as  indicators  that  reveal  the  truth.  Depending  on  the  study,  the 
author  may  use  clues  or  cues.  This  paper  will  refer  to  them  all  as  cues  unless  directly 
quoting  another  author.  Eeakage  or  deception  cues  may  be  displayed  in  a  variety  of 
ways;  a  change  in  facial  expression,  change  in  tone  or  rate  of  speech  or  breathing  pattern, 
verbal  mistakes,  or  long  pauses  between  words  (Ekman,  1985:43). 

When  a  wife  asks  her  husband  if  he  thinks  she  looks  fat  and  he  avoids  eye  contact  and 
insists  that  she  is  not,  he  may  be  demonstrating  leakage.  The  avoidance  of  eye  contact 
suggests  that  he  does  think  she  looks  fat,  but  he  does  not  want  to  say  so.  On  the  other 
hand,  if  she  asks  what  he  wants  for  dinner  and  he  again  avoids  eye  contact  while  saying, 
“anything  you  fix  will  be  wonderful  dear,”  the  avoided  eye  contact  again  may  indicate 
deceit,  but  in  this  case  we  do  not  know  what  he  wants.  He  may  want  something  very 
specific,  or  he  may  just  be  sick  of  the  meatloaf  he  has  had  every  night  for  a  week. 

An  earlier  study  found  that  learning  deception  cues  was  a  matter  of  learning  which 
cues  consistently  distinguish  between  truth  and  deception  (Zuckerman,  Koestner,  and 
Alton,  1984:520-526).  This  study  found  that  training  can  significantly  improve  the 
“human  lie  detector”,  and  when  feedback  was  added  to  practice  sessions,  accuracy 
improved  to  70  percent.  They  also  found  that  despite  fatigue  and  reduced  concentration, 
accuracy  improved  between  the  first  half  of  the  test  to  the  second  half.  They  purport  that 
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the  improvement  is  possibly  due  to  getting  additional  practice  from  the  first  half  of  the 
test  and  what  should  have  been  learned  through  training  was  actually  learned  through 
practice  during  the  test. 

With  various  combinations,  according  to  one  study,  up  to  700,000  deception  cues  are 
possible  (Schrank,  1975:37).  With  that  many  possible  cues,  it  is  extremely  difficult  to 
watch  for  them  all.  Some  examples  of  cues  are  choice  of  words,  expressions  gestures, 
head  movements,  posture,  sweating,  flushing,  etc.  The  most  commonly  watched  cues  are 
the  choice  of  words  and  those  involving  facial  expressions  respectively  (Ekman,  1985:81- 
82).  Since  they  are  the  most  commonly  watched,  they  are  also  the  cues  that  are  the  most 
often  attempted  to  be  controlled.  Because  there  are  many  ways  a  deceiver  may  display 
cues,  this  review  will  categorize  them  into  word  choice,  facial  expressions,  other  verbal 
expressions,  and  other  nonverbal  expressions. 

Word  Choice 

The  deceiver  knows  that  his  words  are  being  listened  to,  and  that  his  choice  of  words 
can  make  a  difference  in  the  success  or  failure  of  his  attempted  deceit  (Ekman,  1985:87- 
91).  If  the  deceiver  has  had  time  to  prepare  and/or  rehearse  his  lie,  he  may  choose  better 
words.  However,  if  he  has  not  had  time  to  prepare  the  lie,  he  is  more  apt  to  be 
inconsistent  in  the  story  told,  use  inappropriate  words  for  the  discussion,  or  repeat  words 
or  phrases,  all  in  an  attempt  to  gain  more  time  to  create  his  fabrication. 

Meyer  (1992:240)  also  breaks  deception  down  into  those  who  have  been  able  to 
practice  and  those  who  have  not.  The  deceiver  who  must  make  up  lies  on  the  fly 
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provides  brief  but  tentative  answers  to  questions,  has  more  silent  pauses,  uses 
meaningless  phrases  such  as  “like  that”  and  “you  know”  as  a  more  polished  verbal  pause, 
uses  more  absolute  words  such  as  “always”  and  “never”,  and  fails  to  provide  important  or 
obvious  details.  Whereas  the  prepared  deceiver  provides  more  immediate  answers  to 
questions  and  those  answers  are  more  often  terse  answers. 

Facial  Expressions 

Second  only  to  chosen  words,  the  face  is  the  most  frequently  watched  and 
manipulated  feature  when  it  comes  to  deception  (Ekman,  1985:123-161).  Some  facial 
cues  are  lip  biting  or  sucking,  cheek  puffing,  feigned  smiles,  and  the  raised  eyebrow. 
Many  facial  expressions  are  easily  controllable,  but  there  are  some  muscles  or  features 
that  most  people  cannot  control.  The  most  common  uncontrollable  muscles  are  in  the 
forehead  and  eyebrows.  Blushing  and  blanching  are  also  features  of  the  face  that  are 
virtually  uncontrollable.  These  features  of  the  face  are  more  likely  to  show  the  deceiver’s 
true  emotions. 

There  are  also  micro  expressions.  A  micro  expression  is  one  that  is  displayed  for  less 
than  a  quarter  of  a  second,  but  it  is  usually  a  full  facial  expression  that  may  briefly  reveal 
a  person’s  true  feelings  (Ekman,  1985:129).  Squelching  is  similar  to  a  micro  expression, 
but  it  only  involves  part  of  the  face  and  it  usually  lasts  longer. 

The  eyes  have  their  own  deceptive  cues.  The  controllable  eye  features  are  the 
direction  of  gaze  of  the  eyes  and  the  muscles  around  the  eyes  (Ekman,  1985:141).  The 
muscles  around  the  eyes  control  the  shape  of  the  eye  and  how  much  of  the  white  and  iris 
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of  the  eye  is  showing.  Some  eye  features  that  are  not  eontrollable  with  eertain  emotions 
are  pupil  dilation,  blinking,  and  tearing. 

Other  Verbal  Cues 

Many  verbal  eues  are  not  neeessarily  one’s  ehoiee  of  words.  Many  verbal  eues  ean 
be  displayed  through  other,  sometimes  uneontrollable,  voiee  idiosynerasies  (Ekman, 
1985:92-98;  Meyer,  1992:240-245).  Some  of  these  eues  ean  be  pauses  (too  long  or 
frequent),  verbal  pauses  sueh  as  “um”  or  “aah”,  stumbling  over  words  (stuttering),  a  rise 
in  the  piteh  of  the  voiee,  a  ehange  in  rate  of  speeeh,  tremors  in  speeeh,  or  flat/monotone 
speeeh  with  no  emotion.  These  behaviors  eould  all  be  cues  to  deception,  or  they  could 
just  be  the  way  the  person  talks.  So  the  key  to  use  of  these  cues  to  detect  deception  is  to 
listen  for  changes  in  the  person’s  normal  speech  patterns. 

Other  Nonverbal  Expressions 

Perhaps  the  most  overlooked  method  of  detecting  deception  is  the  monitoring  of  body 
movements  (Ekman,  1985:98-1 14;  Meyer,  1992:240).  As  previously  stated,  the  most 
watched  areas  are  those  of  choice  of  words  and  facial  expressions.  Since  those  areas  get 
the  most  attention,  deceivers  tend  to  neglect  the  possibility  that  the  rest  of  their  body  may 
be  spewing  deception  cues.  Some  bodily  deception  cues  are  fidgeting,  toe  tapping,  hair 
twirling,  scratching  or  picking,  lip  biting,  covering  the  mouth,  touching  the  nose,  ring 
twisting,  and  concern  over  how  clothes  are  fitting.  The  alert  deceiver  may  be  aware  of 
these  cues  and  take  actions  to  limit  them.  Some  other  cues  that  are  not  as  controllable  are 
changes  in  breathing  patterns,  heart  rate,  swallowing,  and  sweating. 
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In  Schrank’s  (1975:36-41)  studies  of  detecting  deception,  he  posits  that  ninety  percent 
of  all  communication  is  nonverbal.  Hence,  kinesics,  the  language  of  gesture,  is  where 
most  deception  will  be  revealed.  Some  of  the  gestures  he  discusses  are  the  hand-to-chin 
movement,  hands  clasped  behind  the  head,  hair  grooming,  gazing  at  the  ceiling,  rubbing 
the  nose,  forming  a  steeple  with  the  hands,  sitting  with  folded  arms,  and  talking  with  a 
hand  over  the  mouth. 

Meyer  (1992:240),  in  his  separation  of  prepared  and  unprepared  deceivers,  notes  that 
the  unprepared  deceivers  also  tend  to  nod  their  heads  at  the  end  of  a  lie  and  lack 
enthusiasm  indicating  that  they  had  indeed  rehearsed.  Although  it  would  take  hundreds 
of  hours  of  practice  to  become  even  moderately  effective  at  reading  the  estimated 
700,000  possible  combined  gestures,  a  few  hours  of  studying  the  most  obvious  and 
common  gestures  will  aid  one  in  becoming  better  at  detecting  deception  in 
communications  (Schrank,  1975:41).  This  section  has  briefly  discussed  word  choice, 
facial  expressions,  other  verbal  cues,  and  other  non-verbal  expressions  as  indicators  of 
deception.  The  next  section  discusses  how  and  why  those  cues  are  exhibited. 

Categories  of  Cues  Taught 

Burgoon  (in  press)  has  categorized  the  above  cues  into  Arousal,  Emotion,  Cognitive 
Effort,  Memory  Process,  and  Communicator  Tactics.  Tbs  research  will  use  those 
categories  of  cues  to  train  people  on  deception  detection  techniques.  These  categories  of 
cues  use  a  variety  of  the  above-mentioned  cues  as  will  be  described. 


13 


Arousal 


Arousal  is  the  affect  of  the  body  physiologically  reacting  to  excitement,  fear,  or  some 
other  type  of  stimulation,  usually  the  result  of  a  high-stakes  situation  (Burgoon,  in  press). 
The  high-stakes  situation  could  be  an  individual  lying  to  preserve  their  job  or  to  stay  out 
of  jail.  “White  lies”  to  preserve  ones  feelings  are  not  usually  a  high-stakes  situation.  The 
cues  that  will  “leak  out”  of  an  individual  in  a  high-stakes  situation  are  usually  beyond  the 
deceiver’s  control.  Some  examples  of  arousal  cues  are  a  rigid,  still,  stiff,  or  over 
controlled  posture;  dilated  pupils;  lip  pressing;  nervous  self-touches  such  as  nail  biting, 
hair  twirling,  or  picking  on  one’s  own  skin;  clearing  of  the  throat;  coughing;  rise  in 
speaking  pitch;  slowing  of  speaking  tempo;  hesitations  or  restarts  of  sentences,  and  more 
general  nervousness  (Buller  &  Burgoon,  1994:21 1;  Burgoon  et  al,  1992:163;  Burgoon,  in 
press;  Ekman,  1985:114). 

Emotion 

Emotional  cues  are  often  brought  about  by  feelings  of  guilt  or  anxiousness  related  to 
the  fear  of  being  caught  lying.  Although  closely  related  to  arousal,  arousal  can  occur 
without  emotion,  whereas  emotions  must  have  some  degree  of  arousal  (Burgoon,  in 
press).  Some  cues  related  to  emotional  leakage  are  showing  fleeting  or  micromomentary 
expressions  of  fear,  anger,  disgust  or  anxiety;  feigned  smiles;  less  emotion  in  voices; 
monotone  pitch;  less  facial  pleasantness;  less  cooperative  demeanor;  and  forced  or 
nervous  laughter  (Buller  &  Burgoon,  1994:  209;  Burgoon,  in  press;  Ekman,  1985:33-35). 
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Communicator  Tactics 


Evidence  of  communicator  tactics  is  the  result  of  deceivers  employing  strategies  and 
associated  tactics  to  achieve  a  successful  deception.  Unlike  the  other  four  categories  of 
cues,  communicator  tactics  are  purposeful  behaviors  employed  by  the  deceiver  to  achieve 
deception  (Burgoon,  in  press).  The  other  four  categories  examined  in  this  study  typically 
involve  involuntary  or  accidental  behaviors.  Some  of  these  Communicator  Tactics  cues 
are  designed  to  distance  the  deceiver  from  the  story  by  using  equivocation  and  ambiguity; 
also  known  as  nonimmediacy.  The  deceiver  is  trying  to  persuade  his  listener  that  the 
deceiver’s  version  is  truthful.  Therefore,  the  deceiver  is  aware  that  he  will  have  to  use 
different  behaviors  depending  upon  the  situation.  When  the  deceiver  has  had  time  to 
organize  and  plan  out  his  deception,  his  communication  may  have  the  appearance  of 
being  rehearsed,  planned  and  thoughtful.  In  further  attempts  at  this  deception,  he  may 
become  more  assertive  or  dominant  to  exhibit  an  air  of  confidence.  However,  when 
asked  a  question  he  had  not  planned  for,  he  may  use  brevity  in  his  response  to  avoid 
presenting  obviously  false  information.  Hence,  the  deceiver’s  tactics  may  have  an  either 
fight  or  flight  approach.  The  fight  being  more  assertive  and  engaging,  and  the  flight 
being  submissive  or  withdrawn.  In  longer  deceptions,  initial  rigidity  and  over-control  of 
behavior  may  succumb  to  a  more  relaxed  behavior  as  the  deceiver  becomes  comfortable 
in  his  deceit.  Additionally,  initial  unpleasantness  may  succumb  to  increased  pleasantness 
such  as  smiling  and  facial  pleasantness  and  vocal  pleasantness.  Other  indicators  of 
Communicator  Tactics  cues  are  longer  response  latencies,  vocal  uncertainty,  and  less 
nonverbal  immediacy  and  involvement  (Burgoon,  in  press). 
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Memory  Process 


Memory  Process  cues  are  the  result  of  deceivers  having  to  conjure  up  a  story. 
Truthtellers  have  real  memories  and  they  can  relate  rich  details.  Deceivers  must  take  the 
time  to  create  a  story  and  since  it  is  not  real,  the  story  will  generally  lack  the  richness  and 
details  of  a  real  story.  The  reduction  in  details  expressed  by  the  deceiver  corresponds  to 
less  vocalization  or  talk  time  exhibited  by  deceivers.  A  person  accessing  actual 
memories  shows  greater  expressivity  and  punctuated  head  movements,  therefore,  a 
person  conjuring  up  a  story  would  display  less  of  these  characteristics  (Burgoon,  in 
press). 


Cognitive  Effort 

Deception  is  usually  more  difficult  than  telling  the  truth.  This  difficulty  reveals  itself 
through  cognitive  effort  cues.  However,  cognitive  effort  cues  may  be  limited  especially 
if  the  deceiver  has  had  time  to  plan  and  rehearse  the  deception  or  in  the  case  of  written 
deception,  having  the  opportunity  to  edit  content  before  sending  the  message.  Some  cues 
that  would  indicate  cognitive  effort  are  the  deceiver  having  difficulty  coordinating  his 
verbal  and  nonverbal  expressions  (i.e.:  saying  a  food  is  delicious  while  grimacing); 
avoiding  eye  contact;  temporarily  stopping  natural  gestures  when  speaking;  speaking 
slower;  having  more  pauses  and  errors  in  speech;  and  short  and  monotone  responses  to 
questions.  (Schrank,  1975:39;  Burgoon  &  Buller,  1994:176)  Under  high-stake  situations, 
deceivers  will  use  fewer  fdled  pauses  and  non-ah  non-fluencies,  but  under  low-stake 
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situations  deceivers  use  more  filled  pauses  and  non-fluencies  (Burgoon,  in  press).  This 
fluctuation  in  cues  makes  this  a  difficult  category  to  master. 

Use  of  Cues 

Buller  and  Burgoon  (1994:369)  codify  deception  into  three  types  of  deception: 

•  Falsification:  most  conversationally  and  informationally  complete 

•  Equivocation:  least  conversationally  and  informationally  complete 

•  Concealment:  withholding  information 

Their  codification  adds  one  more  category  to  Ekman’s  categories.  Equivocation  is  a 
sort  of  middle  of  the  road  between  concealment  and  falsification.  Equivocation  contains 
no  deviation  from  honesty  (Buller  and  Burgoon,  1990:397),  just  a  lot  of  ambiguity  and 
room  for  misinterpretation.  Although  falsification  is  generally  the  more  studied  form  of 
deception,  equivocation  may  be  the  more  practiced  form  of  deception. 

Buller  and  Burgeon’s  (1994:378-391)  research  found  that  deceivers  generally  are 
more  kinesically  expressive;  nodded  less  yet  had  more  random  movement;  were 
considered  less  formal  and  more  withdrawn,  possibly  in  an  attempt  to  suppress  deception 
cues  and  reduce  the  likelihood  of  being  caught;  and  were  less  pleasant.  Additionally, 
Buller  and  Burgoon  noted  different  behaviors  depending  on  the  relationship  with  the 
receiver.  If  the  receiver  was  a  stranger,  the  deceiver  was  conversationally  less  dominant 
yet  overall  more  dominant,  more  reticent,  displayed  less  kinesic  tension,  and  altered  their 
behavior  less.  Yet,  another  study  found  that  deceivers  swiveled  more  in  their  chairs. 
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spoke  slower,  used  more  present  tense  verbs  and  less  past  tense  verbs,  spoke  longer,  used 
more  group  referenees,  more  modifiers,  and  less  self  referenees  (Ebesu  and  Miller, 
1994:418-442).  Burgoon  and  Buller  (1994:173)  found  that  deeeivers  had  greater 
nonimmediacy  and  reticence,  vagueness  and  uncertainty,  arousal  and  nervousness, 
negative  affect,  and  communication  incompetence.  They  also  avoided  gaze  and  gave 
briefer  answers. 

One  study  has  shown  that  when  individuals  are  first  exposed  to  truthful  behavior, 
deceptive  behavior  is  more  successfully  detected  (Zuckerman,  Koestner,  and  Alton, 
1984:519).  The  deception;  however,  is  generally  detected  through  body  cues  instead  of 
the  usually  watched  facial  cues.  This  earlier  discovery  has  been  further  supported  by 
Burgoon,  et  al  (1994:267).  They  found  that  deviations  from  typical  communication 
patterns  may  be  the  single  best  indicator  of  deception  attempts.  For  example,  although 
speaking  in  a  monotone  voice  is  an  indicator  that  someone  may  be  lying,  some  people 
speak  monotonically  on  a  regular  basis.  Therefore,  looking  for  a  change  in  speech 
pattern  is  a  more  accurate  method  of  determining  whether  that  person  is  being  truthful 
(Zuckerman  et  al,  1984:519;  Feeley  &  Young,  2000:102). 

Burgoon  and  Buller  (1994:160)  codified  deception  into  two  further  categories: 
strategic  and  nonstrategic  communications.  Evidence  of  strategic  communication  is 
vagueness/uncertainty,  withdrawal/reticence/nonimmediacy,  disassociation,  and  image- 
protecting  behavior.  These  strategic  communications  are  also  known  as  information 
management,  behavior  management,  and  defensive  maneuvers  because  they  are  less 
committed  towards  deception;  rather  they  are  an  attempt  to  foster  a  poised,  friendly 


18 


relationship  with  the  reeeiver  to  allay  suspieion.  Nonstrategie  eommunieation  behaviors 
are  more  eommon  deeeptive  displays  sueh  as  leakage,  arousal  and  nervousness  eues, 
negative  affeet  eues,  and  ineompetent  eommunieation  performanees. 

The  interpersonal  deteetion  theory  (IDT)  posits  that  soeially  skilled  people  are  better 
able  to  deeeive  and  avoid  deteetion  (Burgoon,  et  al(b),  1994:289-295).  By  the  same 
token,  IDT  eonveys  that  soeially  skilled  people  are  better  at  deteeting  deeeitful  behavior. 
Soeially  skilled  senders  will  alter  their  behavior  based  upon  their  pereeived  feedbaek 
from  reeeivers.  If  they  feel  they  are  not  being  eompletely  sueeessful  in  their  deeeit,  they 
will  ehange  their  behavior  in  an  attempt  to  deeeive  better.  On  the  other  hand,  soeially 
skilled  reeeivers  will  hide  suspieiousness  in  an  attempt  to  make  the  sender  eomfortable  to 
possibly  aequire  more  deeeptive  eues  or  eontradietory  information. 

The  motivation  impairment  effeet  (MIE)  says  that  when  speakers  are  more  motivated, 
their  verbal  deeeption  skills  will  improve,  but  their  nonverbal  skills  will  be  impaired. 
Burgoon  and  Floyd  (2000:258-262)  eondueted  a  study  whieh  eontradieted  the  MIE 
findings.  They  found  that  eontrary  to  those  previous  findings,  motivation  did  not 
negatively  affeet  nonverbal  performanee  by  deeeivers;  motivation  was  rarely  assoeiated 
with  inhibited  verbal  or  nonverbal  performanee. 

This  literature  review  argues  that  the  aforementioned  eues  are  valid  eharaeteristies  to 
use  in  a  determination  as  to  whether  or  not  deeeption  has  oeeurred.  While  eaeh  eue  is  a 
valid  indieation  of  possible  deeeption,  the  observer  of  the  eue  must  weigh  the  entire 
eommunieation  and  determine  if  expression  of  that  eue  is  out  of  eharaeter  for  the  sender 
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to  make  an  educated  judgment  as  to  whether  or  not  the  sender  is  being  deceptive. 
Learning  these  cues  may  seem  overwhelming,  but  it  is  the  intent  of  this  study  to 
determine  if  CBT  assists  in  this  learning  process. 

Third-Person  Observation 

This  research  examined  the  ability  of  study  participants  to  accurately  evaluate 
whether  or  not  deception  was  employed  between  two  other  individuals  in  a  previously 
recorded  scenario.  As  such,  the  study  participants  are  third-party  observants.  Several 
studies  indicate  significant  differences  in  how  senders,  receivers,  and  observers  interpret 
the  same  interactions  (Burgoon,  et  al,  1994:264;  Guerrero,  L,  1994:125-140).  Since  the 
actors  (sender  and  receiver)  are  involved  in  cognitively  intensive  communication,  they 
may  miss  or  misinterpret  behaviors  of  the  other.  In  general,  the  participants  determine 
the  other’s  communication  to  be  more  favorable  than  an  outside  observer. 

In  one  study,  an  independent  observer  was  watching  a  college  professor  interview  one 
of  his  students  (Ekman,  1985:100).  In  support  of  the  study,  the  instructor  was 
intentionally  making  it  a  stressful  interview.  During  the  interview,  the  student 
unintentionally  and  indiscriminately  gave  the  instructor  the  finger.  When  the  observer 
informed  both  the  professor  and  the  student  what  was  observed,  they  both  denied  that  it 
happened.  When  the  observer  showed  them  a  videotape  of  the  incident,  they  were  both 
shocked.  Not  only  does  this  incident  support  the  third-person  observer,  but  it  also 
supports  theory  that  deceivers  exhibit  cues  unintentionally. 
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Burgoon,  et  al  (1996:735-737)  conducted  an  extensive  study  on  the  issue  of  sender, 
reeeiver,  observer  interpretations  on  oommunieations.  They  found  that  reeeivers  did  not 
reeognize  ehanges  in  sender’s  verbal  message  or  nonverbal  behaviors  when  shifting  from 
truthful  to  deeeptive  messages.  One  of  the  reasons  Burgoon  points  out  why  the  reeeiver 
may  not  notiee  the  shift  from  truthfulness  to  deeeption  is  beeause,  aeeording  to  IDT,  he 
enters  the  eonversation  with  a  very  strong  expeetation  for  honesty.  Another  reason  the 
reeeiver  may  not  notiee  the  shift  is  beeause  he  may  be  too  busy  eoordinating  his  own 
eommunieation.  Planning  eommunieation  involves  extensive  eognitive  effort  whieh  may 
hinder  one’s  ability  to  interpret  an  interaetion. 

The  experiments  for  this  researeh  use  a  third-person  observer  to  make  a  determination 
as  to  whether  or  not  deeeption  was  employed.  The  previous  research  discussed  in  this 
seetion  indieates  the  use  of  third  person  observers  are  better  at  evaluating  individual 
behaviors  than  one  direetly  involved  in  the  situation  (Burgoon,  et  al,  1994;  Guerrero,  L, 
1994;  Ekman,  1985;  Burgoon,  et  al  1996).  The  literature  contains  many  examples  of 
studies  on  use  of  the  third-person  observer,  validating  the  deeision  to  use  a  third-person 
observer  for  this  researeh. 

Computer-Based  Training  (CBT) 

Note  that  much  of  the  literature  reviewed  for  this  researeh  used  the  term  Computer 
Based  Instruetion  (CBl).  For  eonsistency  purposes,  this  paper  will  use  CBT  in  plaee 
CBl.  As  previously  diseussed,  this  researeh  eompared  the  results  of  traditional 
instruction  with  that  of  CBT.  CBT  often  offers  the  learner  more  eontrol  (choiees  in 
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practice  level,  time  spent  on  eaeh  task,  and  level  of  attention  at  partieular  points)  over  the 
learning  proeess  whieh  ean  be  eritieal  a  determinant  of  training  effeetiveness  (Brown, 
2001:271).  While  CBT  may  be  appealing,  some  studies  have  shown  that  giving  the 
learner  eontrol  may  hinder  learning  (Brown,  2001:281).  The  learners  may  skip  eritieal 
praetiee  sessions,  or  move  quiekly  through  training  in  an  attempt  to  get  it  over  with. 
Aeeording  to  Brown,  some  of  the  students  with  the  worst  Post-test  seores  eompleted  less 
than  70  pereent  of  the  praetiee  sessions  and  spent  less  than  six  hours  on  a  14-hour 
program. 

Communieation  riehness  is  the  ability  to  eommunieate  information  while  overeoming 
ambiguity  or  diffieult  ideas  without  losing  the  meaning  of  the  information  (Daft  and 
Lengel,  1986:560).  The  media  the  eommunieation  uses  is  referred  to  as  media  richness. 
The  better  the  media  is  able  to  transfer  the  intended  eommunieation  without  degradation, 
the  rieher  the  media.  Daft  and  Lengel  (1986)  rate  several  types  of  media  in  deereasing 
richness  starting  with  faee-to-faee  eommunieation.  This  is  the  riehest  beeause  it  allows 
immediate  feedbaek  and  observanee  of  body  language,  nonverbal,  and  verbal  eues.  The 
other  media  evaluated  are  telephone,  personal  doeuments,  impersonal  doeuments,  and 
numerie  doeuments.  Sinee  their  study  was  eompleted  prior  to  the  proliferation  of 
eomputers,  CBT  and  CD-ROMs  were  not  evaluated. 

Aeeording  to  a  more  reeent  study,  87  pereent  of  eompanies  with  at  least  100 
employees  have  used  CD-ROMs  for  training  (Kaupins,  2002:319-322).  This  partieular 
study  investigated  the  effeetiveness  of  six  objeetives  that  CBT  was  attempting  to  train. 
Those  objeetives  are  knowledge  aequisition,  problem  solving,  partieipant  aoeeptanee. 
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changing  attitudes,  interpersonal  skills,  and  knowledge  retention.  Television  lectures 
scored  lowest  for  most  objeetives,  while  the  highest  rated  methods  were  role-playing, 
games  and  eonferences.  The  latter  methods  offer  more  of  a  faee-to-faee  type  of 
eommunieation  whieh  Daft  and  Lengel  (1986)  rate  as  highest  in  media  riehness;  henee, 
the  not  so  surprising  results  that  those  methods  were  more  effeetive. 

These  results  support  previous  studies  that  suggested  that  adults  learn  more 
effectively  through  aetive  training  methods  and  applieation  of  knowledge  based  on  real- 
life  events.  Kaupins  (2002)  finds  mixed  results  between  a  1999  study  that  found  no 
signifieant  differences  between  online  and  offline  seetions  of  the  same  courses,  and  a 
2000  study  that  student’s  attitudes  toward  Web  eourses  were  more  favorable  with 
students  who  were  more  experieneed  with  the  Web.  A  2003  study  found  that  training 
via  CBT  resulted  in  at  least  as  good  as  and  slightly  better  than  results  from  live  instruetor 
training  (Lankowski,  2003;  Dzubinski,  2003;  Knode,  2003;  Roekwell;  2003). 

Web  eourses  ean  provide  a  variety  of  partieipation  through  videos,  interaetive  videos, 
online  chats,  and  audio.  Of  the  twenty  different  variations  of  training  tested,  CD-ROMs 
plaeed  ninth,  and  video  plaeed  15**'.  Speeifieally  for  knowledge  aequisition  and  retention, 
CD-ROMs  and  videos  were  rated  relatively  high,  but  lower  for  interpersonal  skills 
development.  The  researeh  eondueted  in  this  study  eompiled  the  use  of  video,  CD-ROM, 
and  slide  presentations  in  the  development  of  the  CBT.  In  the  eontext  of  this  research, 
deception  deteetion  training  would  be  eonsidered  a  mix  between  knowledge  aequisition 
and  retention  and  interpersonal  skills  development. 
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There  are  mixed  results  in  determining  the  effeetiveness  of  CBT  in  the  learning 
proeess.  This  is  a  limitation  that  will  be  discussed  further  in  chapter  5  of  this  research. 

Agent99  Development 

Recent  research  in  the  area  of  deception  detection  has  resulted  in  development  of  the 
Agent99  Trainer,  a  Computer  Based  Training  (CBT)  system  designed  to  be  self-paced 
and  provide  immediate  feedback  to  the  student.  Agent99  is  an  improvement  upon  a 
previously  developed  system  called  Learning  By  Asking  (LBA).  LBA  is  a  general 
training  tool  that  provides  the  flexibility  of  web-based  technology  while  providing  the 
media  richness  of  multimedia  technology.  LBA  provided  explicit  learning  instruction, 
but  lacked  practice  and  feedback;  two  critical  components  of  learning  deception 
detection.  The  three  components  together,  explicit  learning  instruction,  practice,  and 
feedback  are  the  three  core  components  of  learning  deception  detection.  Therefore,  to 
complete  the  triad,  researchers  added  the  “View  Example  with  Analysis”  module  which 
provided  scenarios  that  users  could  practice  with  and  obtain  feedback  on  their  judgments 
as  to  whether  deception  was  employed.  The  addition  of  the  “View  Example  with 
Analysis”  to  the  earlier  designed  EBA  is  the  Agent99  Trainer  (Cao  et  al(b),  2003:2-3). 

Previous  Agent99  Research 

The  previous  study  that  this  research  will  be  continuing  compared  Agent99  with 
instructor  lecture  training  methods.  This  previous  study  used  120  Air  Eorce  Officers  in 
Basic  Communication  Officer  Training  (BCOT)  at  Keesler  Air  Eorce  Base  (George, 
2003:1;  Eankowski,  2003;  Dzubinski,  2003). 
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Each  method  used  identieal  eurriculum  to  ensure  effeetive  examination  of  the  training 
method  without  the  eorruption  of  differing  training  material.  The  eurrieulum,  based  on 
what  was  found  to  be  effective  in  previous  studies,  ineluded  a  videotaped  leeture, 
eommunieation  seenarios,  and  analyses  for  eaeh  seenario  (Cao,  et  al(a).  2003:3). 

The  eurrieulum  as  detailed  above  significantly  improved  deeeption  deteetion  (Cao,  et 
al(a),  2003:7).  While  the  study  showed  no  signifieant  improvement  using  the  Agent99 
Trainer  over  instruetor  lecture  methods,  Agent99  was  proven  to  be  at  least  as  effective  as 
instruetor  leeture  methods.  One  reason  for  the  weaker  than  expeeted  performanee  may  be 
that  the  Agent99  software  was  not  fully  funetional.  An  interactive  option  of  the  Agent99 
Trainer  was  still  in  development  at  the  time  of  the  study.  This  interaetive  option  affords 
the  opportunity  for  students  to  perform  keyword  searehes  and  ask  the  CBT  questions  and 
reeeive  answers  (Cao,  et  al(a).  2003:7). 

Hypotheses 

With  the  Agent99  Trainer  now  fully  funetional,  this  study  examines  the  level  of 
effeetiveness  for  training  of  each  of  the  five  eategories  of  deeeption  eues.  This  study  uses 
the  Agent99  Trainer  to  teaeh  students  how  to  deteet  deeeption  cues.  The  students  are 
tested  on  their  ability  to  detect  cues  first  to  develop  their  baseline  abilities.  They  are  then 
instrueted,  via  Agent99  Trainer,  then  tested  again  to  determine  if  their  deteetion  abilities 
improved.  The  expeetation  is  that  students’  ability  to  detect  deception  will  improve  after 
reeeiving  training  via  the  Agent99  Trainer.  This  seenario  leads  to  the  following  five 
hypotheses: 
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Given  that  people  will  unintentionally  display  certain  deception  cues  due  to  a 
physiological  reaction  to  excitement,  fear,  or  some  other  type  of  stimulation,  there  is  a 
general  expectation  that  those  cues  can  be  taught  and  learned.  The  common  cues 
associated  with  deception  due  to  arousal,  such  as  rigid,  still,  stiff,  or  over  controlled 
posture;  dilated  pupils;  lip  pressing;  nervous  self-touches  such  as  nail  biting,  hair 
twirling,  or  picking  on  one’s  own  skin;  clearing  of  the  throat;  coughing;  rise  in  speaking 
pitch;  slowing  of  speaking  tempo;  hesitations  or  restarts  of  sentences,  and  more  general 
nervousness  (Buller  &  Burgoon,  1994:21 1;  Burgoon  et  al,  1992:163;  Burgoon,  in  press; 
Ekman,  1985:114). 

HI:  Training  via  Agent99  Trainer  will  improve  the  detection  of  Arousal  cues  to 
indicate  that  deception  is  being  employed. 

Since  people  may  have  anxiety  about  lying  or  a  fear  of  getting  caught  lying,  they  may 
display  those  fears  with  emotionally  related  deception  cues.  Some  of  the  common 
emotion  cues  that  would  be  displayed  are  showing  fleeting  or  micromomentary 
expressions  of  fear,  anger,  disgust  or  anxiety;  feigned  smiles;  less  emotion  in  voices; 
monotone  pitch;  less  facial  pleasantness;  less  cooperative  demeanor;  and  forced  or 
nervous  laughter  (Buller  &  Burgoon,  1994:  209;  Burgoon,  in  press;  Ekman,  1985:33-35). 
It  is  expected  that  it  is  possible  to  train  people  to  recognize  these  cues. 

H2:  Training  via  Agent99  Trainer  will  improve  the  detection  of  Emotion  cues  to 
indicate  that  deception  is  being  employed. 
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Purposeful  behaviors  employed  by  a  deeeiver  are  intended  to  eoneeal  that  they  are 
being  deeeptive  or  to  distanee  themselves  from  the  story  (Burgoon,  in  press).  Some 
Communieator  Taeties  eues  that  may  be  displayed  in  the  eourse  of  a  deeeption  are 
aggressive  and  engaging  or  submissive  and  withdrawn  behavior,  terse  responses  to 
questions,  initial  unpleasantness  switehing  to  pleasantness,  more  silent  pauses,  longer 
response  lateneies,  voeal  uneertainty,  and  less  nonverbal  immediaey  and  involvement 
(Burgoon,  in  press).  With  training,  it  is  expeeted  that  individuals  will  be  able  to  learn 
these  eues. 

H3:  Training  via  Agent99  Trainer  will  improve  the  deteetion  of  Communieator 
Taeties  eues  to  indieate  that  deeeption  is  being  employed. 

In  the  proeess  of  making  up  a  story,  deeeivers  generally  do  not  eonvey  detailed 
riehness  that  one  would  eonvey  in  the  eonveyanee  of  a  truthful  situation.  In  eonveying  a 
false  story,  a  deeeiver  may  use  less  voealization  or  shorter  talk  time  due  to  the  faet  that  he 
is  providing  less  detail  than  would  be  provided  from  an  aetual  memory.  He  may  also 
show  less  expressivity  and  redueed  head  movements  during  the  deeeit  (Burgoon,  in 
press).  It  is  expeeted  that  one  eould  learn  to  be  eognizant  of  these  behaviors  with 
training. 

H4:  Training  via  Agent99  Trainer  will  improve  the  deteetion  of  Memory  Proeess 
eues  to  indieate  that  deeeption  is  being  employed. 

The  diffieulty  in  attempting  to  make  deeeption  appear  to  be  truthful  generally  eauses 
diserepaneies  in  the  deeeiver’s  verbal  and  nonverbal  behavior  (i.e.:  saying  a  food  is 
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delicious  while  grimacing).  Some  other  indicators  of  Cognitive  Effort  cues  are  avoiding 
eye  contact;  temporarily  stopping  natural  gestures  when  speaking;  speaking  slower; 
having  more  pauses  and  errors  in  speech;  and  short  and  monotone  responses  to  questions. 
(Schrank,  1975:39;  Burgoon  &  Buller,  1994:176)  One  complication  with  recognizing 
Cognitive  Effort  cues  is  some  of  the  cues  reverse  depending  upon  whether  it  is  a  high  or 
low  stakes  situation  (Burgoon,  in  press).  Despite  the  difficulties,  one  should  be  able  to 
learn  Cognitive  Effort  cues  and  how  to  use  them. 

H5:  Training  via  Agent99  Trainer  will  improve  the  detection  of  Cognitive  Effort  cues 
to  indicate  that  deception  is  being  employed. 

The  constructs  and  measurement  tools  are  defined  in  Eigure  1  below.  The  Agent99 
Trainer  will  be  the  tool  used  to  train  individuals  to  recognize  the  deception  cues  listed  in 
the  independent  variables.  The  ability  to  detect  cues  after  training  is  conducted  is 
dependent  upon  the  effectiveness  of  the  Agent99  Trainer. 
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Independent  variable 


Dependent  variable 


Figure  1:  What  is  being  measured  and  how? 

The  table  indicates  the  deception  cues  of  Arousal,  Emotion,  Communicator 

Tactics,  Memory  Process,  and  Cognitive  Effort  as  independent  variables.  With  training 

via  the  Agent99  Trainer,  an  increase  in  one’s  ability  to  recognize  the  presence  of  these 

cues  is  expected.  Study  participants  should  improve  their  knowledge  of  the  cues  that 

indicate  deception  and  should  be  able  to  put  that  knowledge  into  practice  by  accurately 

assessing  when  deception  has  been  employed. 
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Ill,  Methodology 


The  previous  chapters  described  the  research  problem,  provided  relevant  background 
information,  and  reviewed  previous  research  results  in  the  areas  of  deception  detection, 
third-person  observations,  computer-based  instruction,  and  the  Agent99  Trainer.  This 
chapter  discusses  the  methodology  employed  in  the  experiment  for  this  research. 


Population 

The  experiments  were  conducted  on  an  Air  Force  installation  using  192  Air  Force 
officers  in  initial  training  as  Communications  Officers.  In  support  of  research  being 
conducted  to  evaluate  the  effects  of  media  richness  on  training  deception  detection,  the 
participants  were  divided  into  five  groups.  All  participants  had  at  least  a  bachelor’s 
degree  and  some  may  have  had  advanced  degrees.  Table  1  delineates  the  study  group’s 
demographic  data. 
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Table  1:  Demographics  of  population 


Demographic  Variable 

n 

Gender 

Male 

159 

Female 

37 

Rank 

2Lt 

177 

ILt 

4 

Captain 

5 

Major 

2 

Lieutenant  Colonel 

0 

Civilian 

2 

Unspecified 

6 

Education 

Bachelor’s  Degree 

174 

Master’s  Degree 

14 

Doctoral  Degree 

0 

Unspecified 

6 

A^e 

Average  (years) 

26.7 

Years  in  Communications  Career  Field 

Average  (years) 

2.2 

Research  Methodology 

The  research  method  chosen  for  this  research  was  a  quasi-experimental 
nonrandomized  control  group  pretest-Posttest  design  (Leedy  and  Ormrod,  2001:241).  An 
experiment  was  chosen  to  examine  this  research  over  other  research  options  because  the 
purpose  of  the  study  is  to  determine  if  instruction  causes  an  improvement  in  the  ability  to 
detect  deception.  The  basic  purpose  of  an  experiment  is  to  determine  a  cause  and  effect 
relationship  (Leedy  and  Ormrod,  2001:229).  For  this  research  the  cause  is  training,  the 
effect  is  expected  to  be  an  improvement  in  the  ability  to  detect  deception.  The  students 
were  divided  into  five  distinct  groups  for  a  concurrent  study  which  evaluated  five 
different  levels  of  media  richness  provided  by  the  Agent99  Trainer  which  will  be 
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explained  later  in  detail.  The  experiment  will  be  a  quasi-experiment  beeause  the  group 
seleetion  eould  not  be  eompletely  random  due  to  the  way  the  students’  were  already 
grouped  for  their  elass  assignments  (Leedy  and  Ormrod,  2001:237).  The  students  were 
grouped  based  on  whieh  elass  they  were  assigned  to  for  their  military  training  and  that 
elass  was  determined  by  the  Air  Foree  Personnel  Center  and  when  they  arrived  for 
training.  Sinee  their  elass  assignment  was  beyond  the  eontrol  of  the  experiment,  it  is 
eonsidered  a  “quasi”  random  experiment.  Eaeh  student  was  assigned  a  four-digit  number 
for  traeking  purposes.  The  four-digit  number  was  used  for  eomparing  Pre  and  Posttests 
and  matehing  demographies  to  the  results.  The  first  digit  of  the  four-digit  traeking 
number  indieated  whieh  group  the  student  was  assigned  to. 

Group  one  reeeived  linear  instruetor  lecture-only  training  via  previously  recorded 
video  presented  via  CD-ROM.  The  training  is  linear  in  that  the  student  did  not  have  the 
ability  to  control  the  sequence  of  the  training.  Group  two  received  linear  training  via 
Agent99  Trainer.  The  difference  between  the  training  group  one  and  group  two  received 
is  that  group  two  received  power  point  slide  presentations  along  with  the  lecture. 

Agent99  also  displayed  the  transcript  the  instructor  was  working  from  below  the  video  of 
the  instructor.  Group  three  received  Agent99  non-linear  training  with  the  ability  to  ask  a 
question  (AAQ).  Non-linear  training  indicates  that  the  student  can  control  the  sequence 
of  the  training.  The  AAQ  option  provides  the  student  the  opportunity  to  type  in  a 
question  and  a  built  in  search  engine  provides  appropriate  answer  options.  In  addition  to 
AAQ,  there  is  a  keyword  search  option.  The  keyword  search  option  is  more  of  a  Boolean 
search  and  provides  a  list  of  more  generic  responses.  Figure  2  depicts  a  capture  of  the 
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actual  screen  the  student  sees.  The  bottom  left  hand  comer  of  the  screen  is  where  the 


student  has  the  option  of  selecting  AAQ  or  Keyword  search. 
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In  your  atteir^ts  to  detect  deception,  study  the  actual 
content  of  the  message  or  conversation.  What  type  of 
emotion  is  present  and  does  it  seem  justified,  ^en  the 
context  of  the  message  and  the  speaker?  People  telling  the 
truth  will  often  egress  tiiel  emotional  states  and  feelings 
more  than  will  deceivers. 
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Figure  2:  Screen  shot  of  Agent99  showing  AAQ  and  Keyword  search 


Group  four  received  Agent99  non-linear  training,  with  AAQ,  and  additional  content. 
Group  five  received  Agent99  non-linear  training,  with  AAQ,  with  additional  content,  and 
with  embedded  pop-up  quizzes. 

Each  group  received  different  instmetions  based  on  the  version  of  the  software  they 
were  to  receive.  Groups  one  and  two  were  told  they  would  receive  an  introduction. 
Pretest,  instmetion,  and  then  a  Posttest.  They  were  also  told  approximately  how  long  the 
instmetion  would  last.  Groups  three  through  five  received  additional  instmetion.  They 
were  told  they  had  the  option  to  move  to  different  parts  of  the  lecture  after  the 
introduction  and  Pretest  portions.  They  were  also  told  they  could  ask  a  question  or  insert 
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a  keyword  search,  the  difference  being  that  ask  a  question  used  algorithms  to  find  an 
answer  and  the  keyword  search  was  a  simple  Boolean  logic  search. 

The  cues  covered  in  the  literature  review,  word  choice,  facial  expressions,  other 
verbal  expressions,  and  other  nonverbal  expressions  are  all  employed  in  one  way  or 
another  in  the  five  categories  (Arousal,  Emotion,  Cognitive  Effort,  Memory  Process,  and 
Communicator  Tactics)  examined  for  this  research.  The  students’  ability  to  learn  to 
recognize  these  cues  is  what  was  measured  for  this  study.  Since  all  students  received  the 
same  instruction,  regardless  of  their  group,  they  all  had  equal  opportunity  to  be  exposed 
to  the  categories  of  cues  to  be  evaluation  in  this  research. 

Measurements 

The  experiment  measured  the  initial  ability  of  each  student  to  accurately  detect 
deception.  Two  separate  sessions  were  used  with  a  3  to  7  day  separation  between  the  two 
for  purposes  affiliated  with,  but  beyond  the  scope  of  this  research.  However,  the 
experimental  design  used  for  the  other  study  lends  itself  to  be  used  for  the  purposes  of 
this  study.  In  Session  one,  their  ability  was  measured  by  a  question/answer  knowledge 
test  and  a  judgment  test  where  each  student  viewed  a  recorded  scenario  and  determined 
whether  or  not  deception  was  employed  in  the  scenario.  The  students  were  then 
presented  with  approximately  a  fifty-minute  presentation  on  recognizing  deception  cues. 
As  previously  stated,  each  group  received  the  presentation  in  different  formats.  After  the 
presentation,  each  student  again  took  knowledge  and  judgment  tests.  Between  three  and 
seven  days  later,  depending  on  BCOT  class  schedules,  the  students  received  Session  two. 
Due  to  scheduling  problems,  only  187  students  were  able  to  return  for  the  second  session. 


34 


Session  two  consisted  of  the  same  format  of  knowledge  and  judgment  tests  with 
additional  instruction  between  the  Pre  and  Posttests.  Session  one’s  instructions  consisted 


of  general  deception  detection  theory  and  history  of  the  study  of  deception  detection. 
Session  two’s  instruction  consisted  of  detailed  explanation  of  the  categories  of  cues  and 
ways  to  recognize  deception. 


The  training  received  by  the  participants  was  the  treatment;  tests  were  the 
observations.  All  five  groups  had  an  initial  observation  (O)  to  determine  their  deception 
detection  base-line  knowledge  and  skills.  They  then  received  treatments  (T)  as  indicated 
above.  Table  2  details  the  chronology  of  the  experiment.  (Leedy  and  Ormrod,  2001:238) 


Table  2:  Treatment  and  Observations 


1  All  Groups 

0  1  T 

0  1 

Three  to  Seven  Day  Break 

1  All  Groups 

0 

T 

_2 _ 1 

Measurement  Methods 

The  student’s  ability  to  accurately  assess  whether  or  not  deception  was  being 
employed  was  determined  via  a  set  of  ten  scenarios  presented  to  them  on  the  CBT.  These 
scenarios  consisted  of  simple  text  of  written  discussions  and  two-person  interviews  with 
audio  and  video  or  with  just  audio.  An  example  of  a  textual  scenario  is  shown  in 
Appendix  B.  The  students  examined  the  scenario  and  made  a  simple  judgment  as  to 
whether  or  not  deception  was  employed.  Then  their  accuracy  between  Pre  and  Posttests 
were  compared  to  determine  whether  they  made  any  significant  improvement. 
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Test  Development 

The  test  scenarios  were  developed  by  scholars  at  the  University  of  Arizona  for  use 
within  a  broad  range  of  areas  within  the  deception  detection  arena.  While  this  research  is 
specifically  examining  which  cues  are  easier  trained,  the  tests  used  for  this  research  were 
not  developed  specifically  for  that  purpose.  They  were  developed  for  other  areas  related 
to  deception  detection. 

Permission  to  Conduct  Experiment 

Given  this  study  involved  the  topic  of  deception,  the  experiment  was  reviewed  by  the 
Human  Subjects  Review  board.  An  exemption  to  API  40-402  was  requested  and  granted 
by  the  Air  Force  Research  Laboratory  Human  Effectiveness  Directorate;  approval 
number  F-WR-2003-0081-E. 

Research  Environment 

The  experiment  was  conducted  in  a  single  classroom  in  the  same  building  in 
which  the  BCOT  students  attend  their  military  training.  The  classroom  consisted  of  16 
student  computers  and  one  instructor  computer.  The  student  computers  were  lined  up 
along  three  walls  of  the  classroom  so  that  all  the  students  faced  the  wall  when 
participating  in  the  experiment.  The  classroom  atmosphere  left  much  to  be  desired  in  the 
way  of  taste  as  one  would  expect  in  a  typical  military  training  environment.  It  was 
essentially  a  bare-boned  room  with  only  the  necessities  and  no  esthetically  pleasing 
qualities.  Certain  parts  of  the  classroom  were  uncomfortably  cold  due  to  location  of  air 
conditioning  vents.  The  students  used  computers  that  were  slightly  dated,  yet  fully 
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capable  of  handling  the  software  used  in  the  study.  The  students  were  also  supplied  a 
pair  of  headsets  in  whieh  to  listen  to  the  audio  portions  of  the  experiment. 

Experiment  Administrators 

The  experiment  was  administered  by  two  Air  Force  Institute  of  Technology  (AFIT) 
Master’s  students  and  a  researeher  from  the  University  of  Arizona  (UA).  The  researeher 
was  primarily  present  to  ensure  the  software  ran  smoothly;  he  had  limited  student 
interaction  and  provided  no  mass  instruction  to  the  students.  The  AFIT  students  were 
both  initially  present  for  every  stage,  but  as  software  problems  were  debugged  and  the 
logistics  of  running  the  experiment  beeame  simpler,  they  alternated  presence  in  the 
classroom.  After  it  was  determined  that  the  software  was  running  smoothly,  the  UA 
researeher  departed. 
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IV.  Data  Analysis 


Overview 

The  previous  ehapter  deseribed  the  methodology  used  for  this  researeh.  This  ehapter 
explains  aetual  events  during  exeeution  of  the  experiment,  analyzes  the  quantifiable 
results  of  the  experiment,  and  diseusses  those  results  in  detail. 


Experimental  Events 

The  experiments  eondueted  in  support  of  this  researeh  were  with  the  eooperation  of 
the  BCOT  faeulty,  staff,  and  students.  The  overall  population  size  for  this  experiment 
started  with  192  students  for  the  first  session  and  dropped  to  187  for  the  seeond.  Sinee 
the  students  were  divided  into  five  groups,  there  was  still  eoneern  and  a  struggle  to  obtain 
enough  students  to  ensure  a  large  enough  sample  size  for  eaeh  group.  The  group  sizes 
were  broken  down  as  follows: 


Table  3:  Breakdown  of  Groups  and  Sessions 


Session  1 

Session  2 

Group  1 

32 

30 

Group  2 

44 

44 

Group  3 

30 

30 

Group  4 

45 

42 

Group  5 

41 

41 

Between  Sessions  one  and  two,  two  students  from  group  one  and  three  students  from 
group  four  eould  not  return  for  various  reasons.  Sinee  the  experiment  eonsisted  of  a  Pre 
and  Posttest  during  eaeh  session,  value  was  gained  from  those  five  that  eould  not  return 
for  the  seeond  session.  Therefore,  the  results  of  those  five  individuals  were  not 
disearded. 
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The  students  were  previously  assigned  to  elasses  based  on  their  arrival  date  for 
training.  The  elasses  were  supposed  to  be  limited  in  size  to  sixteen  students;  however,  in 
one  instanee,  eighteen  students  arrived  and  in  another,  seventeen  students  arrived.  Yet,  in 
another  session,  only  two  students  arrived  due  to  lack  of  volunteerism.  Despite  previous 
scheduling  to  ensure  a  smooth  progression  from  group  one  through  group  five,  numerous 
scheduling  conflicts  and  software  problems  arose  that  required  flexibility  in  assignment 
of  students  to  groups.  This  was  necessary  to  ensure  an  adequate  sample  size  for  each 
group.  A  complete  breakdown  of  how  the  students  arrived  follows: 


Table  4:  Numbers  of  students  by  group  and  session 


Session  1 

Session  2  | 

Group  1 

14 

7 

1 

10 

13 

7 

10 

Group  2 

6 

12 

13 

13 

6 

12 

13 

13 

Group  3 

14 

16 

14 

16 

Group  4 

18 

12 

15 

17 

11 

14 

Group  5 

10 

2 

17 

12 

12 

17 

8 

4 

As  students  arrived  for  training,  they  signed  in  using  sequentially  assigned  numbers 
to  maintain  continuity  between  sessions  and  for  documentation  of  demographics.  In  one 
class,  six  students  mistakenly  signed  in  using  a  different  group’s  numbers.  Their 
numbers  were  manually  changed  after  data  collection  to  ensure  validity  between  groups. 
Those  students  were  informed  of  their  number  change  to  ensure  consistency  across 
sessions. 


Data  Analysis 

In  order  to  determine  whether  training  had  any  impact  on  the  students’  abilities  to 
detect  deception,  the  type  of  cues  category  used  in  each  scenario  containing  deception 
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had  to  be  determined.  Sinee  researchers  at  the  University  of  Arizona  had  previously 
constructed  the  scenarios,  their  determination  of  which  cues  were  displayed  in  each 
deceptive  scenario  was  used.  Table  5  below  indicates  which  Pre  and  Posttest  questions 
each  category  of  cues  was  used  in.  Prel  and  Postl  indicate  the  tests  in  Session  one  and 
Pre2  and  Post2  indicate  the  tests  in  Session  two. 


Table  5:  Which  test  questions  were  the  cues  in? 


Pre1 

Postl 

Pre2 

Post2 

Arousal 

1 

4 

1,4,6,10 

3 

Emotion 

2,5 

Tactics 

mmiEKKm 

7,8,9 

1,4,7,10 

Mem  Process 

9,10 

6,7 

2,4,5,6,8,9 

Cognitive 

Effort 

1 

7,8 

5 

The  table  reveals  some  discrepancies  in  the  distribution  of  cues  among  the  Pre  and 
Posttests.  For  example,  in  Session  two.  Arousal  was  presented  in  four  Pretest  questions, 
but  in  only  one  Posttest  question  for  Session  two.  There  were  no  Poshest  questions  that 
employed  Emotion  as  a  deceptive  cue  in  either  of  the  sessions.  Memory  Process 
deception  cues,  in  Session  one,  were  in  two  Pretest  and  five  Poshest  questions,  and  in 
Session  two,  in  two  Pretest  and  six  Poshest  questions.  Cognitive  Effort,  in  Session  one, 
was  in  one  Pretest  and  zero  Poshest  questions,  and  in  Session  two,  two  Pretest  and  only 
one  Poshest  question.  These  discrepancies  were  due  to  the  fact  that  the  questions  were 
not  developed  specifically  for  the  purposes  of  this  research,  but  were  developed  for  use  in 
a  variety  of  other  deception  detection  studies  not  examined  in  this  study. 
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Average  Cue  Score 

Tables  6  and  7  further  show  the  diserepancy  in  the  distribution  of  the  eues  in  Session 
one’s  Pre  and  Posttests  respectively.  These  tables  also  show  the  average  score  obtained 
by  the  192  students  on  each  question  by  cue.  The  highlighted  scores  indicate  those 
scenarios  where  the  communication  was  truthful. 


Table  6:  Session  One  Pretest 


Question 

Number 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

Arousal 

0.363 

Emotion 

0.574 

0.247 

0.600 

0.637 

Tactics 

0.532 

0.700 

0.726 

0.721 

0.637 

Mem 

Process 

0.637 

0.505 

Cognitive 

Effort 

0.363 

Table  7:  Session  One  Posttest 


Question 

Number 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

Arousal 

Emotion 

Tactics 

Mem 

Process 

Cognitive 

Effort 

0.489 

0.821 

0.679 

0.626 

0.768 

0.879 

0.442 

0.142 

0.505 

Tables  8  and  9  below  show  the  discrepancy  in  the  distribution  of  the  cues  for  Session 
two  in  the  Pre  and  Posttests  respectively.  These  tables  also  show  the  average  score 
obtained  by  the  187  students  on  each  question  by  cue. 
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Table  8:  Session  Two  Pretest  Scores 


Question 

Number 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

Arousal 

0.724 

0.876 

0.643 

0.486 

Emotion 

0.432 

0.795 

Tactics 

0.784 

0.876 

0.784 

0.708 

0.486 

Mem 

Process 

0.643 

0.665 

Cognitive 

Effort 

0.665 

0.784 

Table  9:  Session  Two  Posttest  Scores 


Question 

Number 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

Arousal 

0.314 

Emotion 

Tactics 

0.67 

0.854 

0.957 

0.205 

Mem 

Process 

0.276 

0.854 

0.789 

0.714 

0.459 

0.919 

Cognitive 

Effort 

0.789 

The  average  scores  from  each  Pre  and  Posttest  were  again  averaged  to  obtain  an 
average  score  for  each  cue  on  each  Pre  and  Posttest.  For  example,  the  six  Posttest  scores 
from  Memory  Process  in  Session  two  were  averaged  to  obtain  one  mean  score  for 
Memory  Process  on  Posttest2.  The  Pre  and  Posttest  averages  were  then  compared 
individually  for  each  session  to  obtain  a  delta.  Table  10  below  indicates  the  average 
scores  for  each  cue  by  Pre  and  Posttest. 
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Table  10:  Average  score  by  cue 


Pre1 

Post1 

Pre2 

Post2 

Arousal 

0.3632 

0.4895 

0.6955 

0.3135 

Emotion 

0.5145 

0.6135 

Tactics 

0.6632 

0.7088 

0.7276 

0.6716 

Mem  Process 

0.5711 

0.5474 

0.6541 

0.6685 

Cognitive  Effort 

0.3632 

0.7243 

0.7892 

As  discussed  earlier,  there  are  no  Posttest  scores  for  Emotion  in  either  session  and  no 
Posttest  score  for  Cognitive  Effort  in  Session  one. 


Deltas  between  Pre  and  Posttest 

Once  the  averages  were  computed  for  each  cue  in  each  Pre  and  Posttest,  the 
differences  were  then  computed  to  discover  whether  the  students’  abilities  to  detect 
deception  improved  after  training.  The  differences  are  shown  in  Table  1 1  below.  There 
is  no  difference  in  Emotion  because  the  Emotion  cue  was  not  present  in  any  Posttests. 
There  is  no  difference  indicated  for  Cognitive  Effort  in  Session  one  because  this  type  of 
cue  was  not  present  in  the  Session  one  Posttest. 


Table  11:  Delta  between  Pre  and  Posttests  per  session 


Session 

1 

Session 

2 

Arousal 

0.1263 

-0.3820 

Emotion 

Tactics 

0.0456 

-0.0559 

Mem 

Process 

-0.0237 

0.0144 

Cognitive 

Effort 

0.0649 

The  deltas  represented  above  are  within  each  session,  not  across  sessions.  As  shown, 
in  Session  one,  the  ability  to  detect  Arousal  and  Tactics  cues  increased  slightly,  while  the 
ability  to  detect  Memory  Proeess  cues  decreased  slightly.  In  Session  two,  the  opposite 
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occurred.  The  ability  to  detect  Arousal  and  Tactics  cues  decreased  after  training,  while 
the  ability  to  deteet  Memory  Proeess  and  Cognitive  Effort  ones  inereased  slightly. 


Performing  the  same  analysis  without  the  truthful  questions  yields  only  slightly  better 
results;  however,  it  is  a  more  valid  analysis  since  this  researeh  is  not  attempting  to 
determine  false  positives,  only  the  ability  to  determine  if  training  improves  the  overall 
aeeuracy  in  the  ability  for  students  to  deteet  deeeption.  Table  12  below  shows  the  deltas 
when  the  truthful  seenarios  are  disregarded. 

Table  12:  Deltas  disregarding  truthful  scenarios 


Session 

1 

Session 

2 

Arousal 

0.1263 

-0.3676 

Emotion 

Tactics 

0.0263 

-0.1514 

Mem 

Process 

0.1351 

Cognitive 

Effort 

0.0649 

In  Session  one  Arousal  and  Taeties  again  had  a  slight  improvement  after  training. 
Without  the  truthful  seenarios,  there  were  no  pretest  questions  for  Memory  Proeess  and 
no  Posttest  questions  for  Cognitive  Effort.  In  Session  two.  Arousal  and  Taeties  again 
showed  a  slight  deerease  in  ability  to  deteet  deeeption  and  Memory  Proeess  and 
Cognitive  Effort  showed  a  slight  inerease. 

The  most  eonsistent  results  were  obtained  by  eomparing  the  Session  one  Posttest  with 
the  Session  two  pretest  with  the  truthful  seenarios  disearded.  Table  13  below  shows 
these  results. 
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Table  13:  Delta  between  Postl  and  Pre2  with  trntbfnl  scenarios  discarded 


Arousal 

0.192 

Emotion 

Tactics 

0.079801 

Mem 

Process 

0.106686 

Cognitive 

Effort 

With  this  comparison,  all  measured  eues  showed  at  least  a  slight  improvement.  There 
were  no  data  for  Emotion  and  Cognitive  Effort  due  to  no  Pre  and/or  Posttest  results. 

A  combined  across  the  board  look  at  the  deltas  for  eaeh  cue  yields  eonflieting  results. 
Table  14  below  shows  each  cue  and  the  delta  between  Sessions  one  and  two  and  between 
Session  one’s  Posttest  and  Session  two’s  Pretest. 

Table  14:  Consolidated  Deltas  with  trntbfnl  scenarios  discarded 


Session 

1 

Session 

2 

Postl  - 
Pre2 

Arousal 

0.1263 

-0.3676 

0.192 

Emotion 

Tactics 

0.0263 

-0.1514 

0.079801 

Mem 

Process 

0.1351 

0.106686 

Cognitive 

Effort 

0.0649 

The  only  cues  with  measurements  across  all  fields  are  Arousal  and  Tactics.  These 
two  categories  of  eues  show  improvement  in  the  Pre  and  Posttests  for  Session  one  and 
between  sessions  Post  and  Pretest,  but  show  a  weakening  between  Session  two  Pre  and 
Posttest.  Memory  Proeess  shows  improvement  in  Session  two  Pre  and  Posttest  and 
between  Sessions  one  and  two  Post  to  Pretest.  Cognitive  Effort  only  has  one 
measurement,  the  Pre  and  Posttest  for  Session  two,  whieh  shows  improvement.  Analysis 
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of  Variance  (ANOVA)  tests  were  eondueted  on  eaeh  set  that  are  eomparable  and  are 
included  in  Appendix  B. 

Results  for  HI 

HI:  Training  via  Agent99  Trainer  will  improve  the  deteetion  of  Arousal  eues  to 
indicate  that  deeeption  is  being  employed. 

The  ability  for  students  to  deteet  Arousal  eues  that  indieated  deeeption  in  the 
seenarios  tested  fluctuated.  Between  the  Pre  and  Posttest  in  Session  one,  the  students 
average  ability  to  deteet  deeeption  improved  by  approximately  12.6%.  An  analysis  of 
varianee  (ANOVA)  test,  in  appendix  B,  shows  the  differenee  is  signifieant  with  p  = 
0.0127.  The  most  logieal  explanation  for  the  inerease  in  the  seore  is  that  the  students 
learned  from  the  training  how  to  better  deteet  Arousal  deeeption  cues.  However,  the 
possibility  exists  that  part  of  the  improvement  eould  be  due  to  more  obvious  eues  in  the 
Posttest  than  in  the  Pretest.  From  Session  one’s  Posttest  to  Session  two’s  Pretest,  their 
ability  to  deteet  Arousal  eues  that  indieate  deeeption  improved  again  by  about  19.2%. 
Sinee  there  is  no  training  performed  between  Session  1  Posttest  and  Session  2  Pretest,  the 
improvement  is  a  little  more  suspeet.  However,  the  training  applied  before  Session  1 
Posttest  eould  partially  explain  the  improvement.  There  may  have  been  a  delayed  effeet 
in  ability  to  deteet  Arousal  eues.  But  again,  some  of  the  improvement  eould  be  explained 
on  the  possibility  that  Session  2  Pretest  displayed  more  obvious  eues  than  Session  1 
Posttest.  However,  the  final  measurement.  Session  two’s  Pretest  to  the  Posttest,  showed 
their  ability  to  detect  deception  based  on  Arousal  cues  deereased  by  36.7%.  Although 
elearly  signifieant,  the  ANOVA  shows  this  at  p  =  0.001.  This  large  of  a  deerease  in  the 
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ability  to  detect  Arousal  cues  contaminates  the  previous  improvements.  The  decrease 
could  very  well  be  due  to  the  fact  that  there  was  only  one  question  in  the  Posttest 
compared  to  two  in  the  Pretest.  If  that  one  question  in  the  Posttest  presented  less  obvious 
cues,  with  that  being  the  only  question  to  compare  to,  may  explain  the  degradation  in 
performance.  This  much  of  a  decrease  dwarfs  the  rather  smaller  increases  achieved 
earlier  in  the  experiment,  however  the  limited  comparison  makes  the  degradation 
plausible.  Therefore,  this  study  finds  that  HI  is  partially  supported. 

Results  for  H2 

H2:  Training  via  Agent99  Trainer  will  improve  the  detection  of  Emotion  cues  to 
indicate  that  deception  is  being  employed. 

Only  one  test  scenario  where  deception  was  employed  displayed  the  Emotion  cues. 
Due  to  insufficient  distribution  of  cues  across  the  tests,  not  enough  results  were  collected 
for  measurement  of  the  ability  to  detect  deception  based  on  Emotion  cues.  Therefore, 
this  study  finds  no  evidence  to  support  H2. 

Results  for  H3 

H3:  Training  via  Agent99  Trainer  will  improve  the  detection  of  Communicator 
Tactics  cues  to  indicate  that  deception  is  being  employed. 

The  ability  for  students  to  detect  Communicator  Tactics  cues  that  indicated  deception 
in  the  scenarios  tested  fluctuated.  Between  the  Pre  and  Posttest  in  Session  one,  the 
students  average  ability  to  detect  deception  improved  by  approximately  2.6%.  Although 
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not  a  significant  improvement  (p  =  0.463),  it  does  show  some  improvement.  Sinee  both 
the  Pre  and  Poshest  eonsisted  of  two  questions,  the  most  likely  reason  for  the  inerease  is 
due  to  learning  that  took  place  between  the  tests.  There  is  still  the  possibility  that  there 
was  some  improvement  due  to  more  obvious  eues  in  the  Poshest.  From  Session  one’s 
Poshest  to  Session  two’s  Pretest,  their  ability  to  deteet  Communieator  Taeties  eues  that 
indieate  deeeption  improved  again  by  about  7.9%.  Some  of  this  improvement  eould  be 
explained  by  residual  learning  from  Session  one’s  training,  but  some  may  be  explained 
by  more  obvious  cues  in  Session  two’s  Pretest.  However,  in  the  final  measurement. 
Session  two’s  Pretest  to  the  Poshest,  showed  their  ability  to  detect  deception  based  on 
Communieator  Taeties  eues  deereased  by  15%.  This  is  a  signifieant  differenee  (p  = 
0.001);  eompared  to  Session  one’s  Pre/Post  tests  difference.  Session  two’s  Poshest 
eonsisted  of  two  deeeptive  seenarios,  while  the  Pretest  eonsisted  of  four  deeeptive 
scenarios.  This  uneven  eomparison  may  explain  the  degradation  in  ability  to  deteet 
deeeption  based  on  Communieator  Taeties  cues  if  the  two  scenarios  in  the  Poshest  were 
less  obvious  than  the  four  in  the  Pretest.  Sinee  the  only  improvement  attributable  to 
instruction  resulted  in  no  significant  improvement  and  there  was  signifieant  degradation 
in  ability  to  deteet  deeeption  in  Session  two,  this  study  finds  that  H3  is  not  supported. 

Results  for  H4 

H4:  Training  via  Agent99  Trainer  will  improve  the  deteetion  of  Memory  Proeess 
cues  to  indieate  that  deeeption  is  being  employed. 
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The  ability  to  detect  deception  based  on  Memory  Process  cues  was  measured  in  two 
of  the  three  possible  measurements.  Between  Session  one’s  Posttest  and  Session  two’s 
Pretest,  the  students’  abilities  to  detect  deception  using  Memory  Process  cues  improved 
by  approximately  10%.  Since  there  was  training  conducted  before  Session  one’s  Pretest, 
their  may  have  been  some  residual  learning  that  may  have  caused  the  increase  in  ability 
to  detect  deception  based  on  Memory  Process  cues.  Between  Session  two’s  Pre  and 
Posttest,  the  ability  to  detect  deception  based  on  Memory  Process  cues  increased  by 
13.5%.  ANOVA  tests  show  this  as  significant  with  p  =  0.004.  The  most  likely  cause  of 
the  improvement  is  due  to  training,  however  an  easier  Posttest  cannot  be  ruled  out. 
Because  this  research  shows  two  consistent  significant  improvements  at  the  tail  end  of 
the  experiment,  sufficient  evidence  exists  to  partially  support  H4. 

Results  for  H5 

H5:  Training  via  Agent99  Trainer  will  improve  the  detection  of  Cognitive  Effort 
cues  to  indicate  that  deception  is  being  employed. 

Due  to  limited  distribution  of  cues  across  the  tests,  limited  results  were  collected  for 
measurement  of  the  ability  to  detect  deception  based  on  Cognitive  Effort  cues.  Between 
Session  one’s  Pretest  and  Session  two’s  Pretest,  there  was  a  36%  improvement  in  the 
ability  to  detect  deception  based  on  Cognitive  Effort  cues.  ANOVA  test  show  this 
improvement  at  significant  with  p  =  0.001 .  The  improvement  may  be  explained  by  the 
training  that  took  place  during  Session  one,  but  it  may  be  partially  explained  by  more 
obvious  cues  in  Session  two’s  Pretest.  Between  Session  two’s  Pre  and  Posttests  there 
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was  a  6.5%  increase  in  the  ability  to  detect  deception.  ANOVA  tests  show  this  as  elose 
to  signifieant  with  p  =  0.082.  The  improvement  may  be  due  to  training,  but  more  obvious 
eues  in  the  Posttest  eannot  be  ruled  out.  However,  since  the  Pretest  had  two  deeeptive 
seenarios  and  the  Posttest  had  only  one  deeeptive  seenario,  providing  more  limited 
exposure  to  eues  during  the  Posttest,  it  is  more  likely  that  the  training  had  a  signifieant 
effeet.  Sinee  there  were  two  improvements  in  the  ability  to  deteet  deeeption  based  on 
Cognitive  Effort  cues,  this  study  finds  some  evidence  to  support  H5. 
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V.  Discussion,  Conclusions  and  Recommendations 


The  previous  chapter  revealed  the  quantifiable  results  of  the  experiment  and 
examined  those  results  in  detail.  In  this  chapter,  conclusions,  recommendations, 
implications  for  researchers  and  managers,  and  suggestions  for  future  research  are 
discussed  based  on  the  results  obtained  from  this  experiment. 


Discussion 

The  overall  results  of  this  research  show  promise.  The  lack  of  evenly  distributed 
categories  of  cues  among  the  questions  and  the  variability  between  session’s  Pre  and 
Posttests  cause  difficulties  in  conclusions  of  the  results.  However,  taking  that  limitation 
into  account  permitted  a  logical  diagnosis  of  the  experiment’s  results.  Those  difficulties 
will  be  discussed  later  in  this  chapter. 

Table  15  below  displays  the  results  of  the  hypotheses  in  a  consolidated  format. 


Table  15:  Consolidated  hypothesis  results 


Hypothesis 

Supported 

HI:  Arousal 

Partial 

H2:  Emotion 

No 

H3:  Communicator  Tactics 

No 

H4:  Memory  Process 

Partial 

H5:  Cognitive  Effort 

Supported 

Hypotheses  one  and  four  were  partially  supported,  hypothesis  two  and  three  lacked 
sufficient  evidence  for  support,  and  hypothesis  five  was  considered  supported.  Simply 
stated,  this  study  indicates  that  Arousal  and  Memory  Process  cues  may  be  able  to  be 
taught;  Cognitive  Effort  cues  are  most  likely  able  to  be  taught;  Communicator  Tactics 
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appears  as  though  it  can  not  be  taught,  and  further  study  is  required  to  make  a 
determination  on  Emotion  cues.  Measurements  were  made  where  possible,  particularly 
when  comparisons  between  Pre  and  Posttests  were  not  possible  or  were  weak  due  to 
multiple  data  points  in  one  compared  to  more  limited  data  points  in  another.  In  these 
cases  it  was  determined  that  measurements  across  sessions  would  add  value  to  the 
research  being  conducted.  Extremely  conservative  interpretations  of  the  results  of  this 
study  have  been  made  so  as  not  to  erroneously  link  positive  results  to  the  training 
conducted.  While  the  training  may  have  had  an  impact  on  or  may  actually  be  the  sole 
reason  for  improvements  in  students’  abilities  to  detect  deception,  more  rigid 
experimentation  is  needed  to  unequivocally  establish  that  as  fact. 

Limitations 

The  largest  limitation  of  this  study  is  the  lack  of  evenly  distributed  cues  category 
across  the  Pre  and  Posttest  questions.  The  uneven  distribution  spawns  uncertainty  about 
the  validity  of  the  results  of  the  study.  Another  limitation  is  this  experiment  was 
conducted  with  entry-level  AE  officers  in  one  career  field.  This  significantly  limits  the 
generalizability  of  the  research  results  to  the  rest  of  the  AE  and  Department  of  Defense. 
A  third  limitation  is  the  manner  in  which  the  research  participants  were  obtained. 
Although  it  was  on  a  strictly  volunteer  basis,  new  members  in  the  military  often  feel  that 
volunteering  is  mandatory.  This  expectation  to  volunteer  may  have  led  some  students  to 
volunteer  when  they  really  were  not  interested.  The  students  that  did  not  volunteer  were 
able  to  use  their  time  preparing  for  their  next  exam  or  completing  an  assignment. 
Therefore,  the  volunteers  may  not  have  had  their  minds  entirely  on  learning  deception 
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cues,  but  rather  on  how  they  could  better  be  using  their  time.  Lastly,  several  students 
eommented  that  they  did  not  know  if  they  answered  eorreetly.  Agent99  is  supposed  to 
provide  immediate  feedback,  one  of  the  eore  components  to  learning,  to  students  to  assist 
them  in  learning  deeeption  eues.  Agent99  needs  to  be  reeonfigured  to  ensure  that 
students  get  the  immediate  feedbaek  that  Agent99  is  supposed  to  provide. 

Implications  for  Researchers 

Although  deception  detection  studies  are  in  abundance,  there  is  still  a  long  way  to  go 
in  determining  all  the  uses  and  implications.  Future  researchers  should  continue  studying 
the  cues  categories  examined  in  this  research.  More  carefully  designed  testing  scenarios 
should  be  developed  to  further  validate  the  results  discovered  in  this  research.  The 
Agent99  Trainer  should  continue  to  be  evaluated  for  effectiveness  and  to  determine 
necessary  changes  to  improve  its  performance.  Study  participants’  comments  (Appendic 
C)  should  be  reviewed  and  considered  for  improvements  to  the  Agent99  Trainer. 

Implications  for  Managers 

The  results  of  this  study  should  be  used  by  managers  to  determine  where  to  direct 
resources  to  train  deception  detection.  Based  on  the  results  of  the  experiments,  managers 
should  place  most  of  their  efforts  toward  Cognitive  Effort  cues  because  those  were 
determined  to  be  the  most  trainable  cues.  Secondary  efforts  should  be  place  in  training 
of  Arousal  and  Memory  Process  deception  cues  to  improve  employee’s  ability  to  detect 
deception.  Managers’  use  of  deception  detection  abilities  could  prove  useful  in 
counseling  subordinates  and  in  brokering  business  deals. 
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Conclusions  and  Recommendations 


It  is  the  conclusion  of  this  researcher  that  three  of  the  five  hypotheses  examined  in 
this  study  had  at  least  some  evidence  to  support  the  finding  of  an  improvement.  The 
study  showed  that  the  ability  to  detect  deception  based  on  Arousal  cues  improved  in  two 
of  the  three  measurements  taken.  The  ability  to  detect  Memory  Process  and  Cognitive 
Effort  cues  improved  in  both  sections  measured  for  each.  The  ability  to  detect  deception 
based  on  Communicator  Tactics  showed  insignificant  improvement  in  session  one  and 
significant  degradation  in  session  two.  There  was  insufficient  data  to  determine  if  the 
ability  to  detect  deception  based  on  Emotion  cues  improved  after  training  with  Agent99. 

The  tests  used  in  this  study  were  not  created  for  the  specific  purpose  of  measuring  an 
increase  in  the  ability  to  detect  the  five  cues  selected  for  this  research.  The  unequal 
distribution  of  the  categories  of  deception  cues  among  the  test  questions  severely 
hampered  this  research.  It  limited  the  ability  to  make  comparisons  between  Pre  and 
Posttests  and  voided  the  use  of  more  quantitative  statistical  analysis.  It  is  recommended 
that  future  examination  of  these  cues  use  scenarios  and  test  questions  created  with  the 
intent  purposes  of  comparing  test  results  across  the  five  cue  categories.  This  would 
include  ensuring  that  the  cues  were  evenly  distributed  across  the  test  scenarios  that 
actually  involved  deception  and  ensuring  that  the  cues  depicted  in  those  scenarios  were 
presented  with  consistent  clarity. 

Suggestions  for  Further  Study 

This  researcher  recommends  further  study  in  this  area  of  deception  detection.  The 
identification  of  types  of  cues  can  be  a  valuable  tool  in  determining  where  to  place  efforts 
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and  resources  in  teaching  deception  detection.  If  the  suggestions  above  are  implemented, 
further  research  should  yield  more  valuable  results.  Additionally,  further 
experimentation  should  be  expanded  to  include  more  experienced  leaders  within  the 
military.  The  use  of  entry-level  officers  may  limit  the  results  of  the  study  due  to  a  lack  of 
motivation  to  learn  the  deception  detection  techniques  suggested  in  the  curriculum.  It  is 
recommended  to  apply  the  research  to  senior  level  curriculum  where  senior  level  officers 
would  see  a  value  in  learning  the  deception  detection  techniques  taught.  This  could  be 
done  for  example  at  Squadron  Commander’s  School  or  Air  War  College. 

Summary 

The  use  of  deception  to  achieve  goals  has  been  used  since  the  serpent  deceived  Eve 
into  eating  the  forbidden  fruit  (Zondervan,  I984:Ge2:I0).  Deception  certainly  has  its 
place  in  military  strategy  and  therefore  it  is  critical  that  our  military  leaders  develop  a 
sense  of  when  deception  is  being  employed  against  them  and  how  to  use  it  effectively  to 
gain  a  military  advantage  against  the  enemy  (Friedrich,  1991:39-41;  Schwarzkopf, 
1992:362,  408;  Camor,  2002:426).  This  study  examined  the  effectiveness  of  training 
relatively  new  constructs  in  the  area  of  deception  detection  to  mostly  new  junior  Air 
Force  Officers.  These  constructs.  Arousal,  Emotion,  Cognitive  Effort,  Memory  Process, 
and  Communicator  Tactics,  if  effectively  learned  would  permit  our  military  leaders  to 
better  determine  when  deception  is  being  employed  against  them  (Burgoon,  in  press). 

This  study  determined  that  at  least  four  of  those  five  constructs  are  trainable  to  our  future 
military  leaders.  While  the  overall  study  yielded  somewhat  limited  support  for  the  five 
hypotheses  examined,  this  paper  should  be  used  for  the  improved  development  of  more 
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useful  testing  seenarios  to  further  examine  the  effeetiveness  of  training  of  the  five 
deeeption  deteetion  eategory  eues. 
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Appendix  A:  Analysis  of  Variance  (AN OVA) 

Oneway  Analysis  of  Arousal  Session  1  values  By  Arousal  Pre/Post  1 

1.1 
1 

0.9 
0.8 
c«  0.7 

I  0.6 

§  0.5 
I  0.4 
^  0.3 
0.2 
0.1 
0 

-0.1 


Ar  Prel 


Ar  Posti 


Ar  Pre/Posti 


Oneway  Anova 
Summary  of  Fit 


Rsquare 

Adj  Rsquare 

Root  Mean  Square  Error 

Mean  of  Response 

Observations  (or  Sum  Wgts) 

t  Test 

Assuming  equal  varianees 
Differenee 


Estimate 
Std  Error 
Lower  95% 
Upper  95% 


0.126316 

0.050456 

0.027106 

0.225526 


UtrEqual  Varianees 

Differenee 


Estimate 
Std  Error 
Lower  95% 
Upper  95% 


0.126316 

0.050456 

0.027105 

0.225526 


Analysis  of  Varianee 


0.01631 

0.013708 

0.491787 

0.426316 

380 


t  Test 
2.503 


DF 

378 


Prob  >  I 
0.0127 


t  Test 
2.503 


DF 

377.435 


Prob  >  I 
0.0127 


Souree 

DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Prob  >  F 

Ar  Pre/Post  1 

1 

1.515789 

1.51579 

6.2674 

0.0127 

Error 

378 

91.421053 

0.24185 

C.  Total 

379 

92.936842 

Means  for  Oneway  Anova 

Level 

Number 

Mean  Std  Error 

Lower  95% 

Upper  95% 

Ar  Posti 

190 

0.489474  0.03568 

0.41932 

0.55963 

Ar  Prel 

190 

0.363158  0.03568 

0.29301 

0.43331 

Std  Error  uses  a  pooled  estimate  of  error  varianee 
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Oneway  Analysis  of  Tac  1  values  By  Tac  Pre/Postl _ 

1.1^ - 

1- 
0.9- 
0.8- 
w  0.7- 

CD  ^ ^ _ 

^  0.6- 
ro 

>  0.5- 
o  0.4- 

CO 

>-  0.3- 
0.2- 
0.1- 
0- 

-O.lJ - ^ - 

TacPostI  TacPrel 


Tac  Pre/Postl 


Oneway  Anova 
Summary  of  Fit 


Rsquare 

Adj  Rsquare 

Root  Mean  Square  Error 

Mean  of  Response 

Observations  (or  Sum  Wgts) 

t  Test 

Assuming  equal  variances 
Difference 


Estimate 
Std  Error 
Lower  95% 
Upper  95% 


0.026316 

0.035807 

-0.04409 

0.096722 


UtrEqual  Variances 

Difference 


Estimate 
Std  Error 
Lower  95% 
Upper  95% 


0.02632 

0.03581 

-0.04409 

0.09672 


Analysis  of  Variance 


0.001427 

-0.00121 

0.349005 

0.639474 

380 


t  Test 
0.735 


DF 

378 


Prob  >  I 
0.4628 


t  Test 
0.735 


DF 

376.288 


Prob  >  I 
0.4628 


Source 

DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Prob  >  F 

Tac  Pre/Postl 

1 

0.065789 

0.065789 

0.5401 

0.4628 

Error 

378 

46.042105 

0.121805 

C.  Total 

379 

46.107895 

Means  for  Oneway  Anova 

Level 

Number 

Mean  Std  Error 

Lower  95% 

Upper  95% 

Tac  Postl 

190 

0.652632  0.02532 

0.60285 

0.70242 

Tac  Prel 

190 

0.626316  0.02532 

0.57653 

0.67610 

Std  Error  uses  a  pooled  estimate  of  error  variance 
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Oneway  Analysis  of  Ar  2  Values  By  Ar  Pre/Post2 


w 

CD 

CD 

> 

CNI 

< 


1.1^ 

1- 

0.9- 

0.8- 

0.7- 

0.6- 

0.5- 

0.4- 

0.3-- 

0.2- 

0.1- 

0- 

-0.1- 


Ar  Post2 


Ar  Pre2 


Ar  Pre/Post2 


Oneway  Anova 
Summary  of  Fit 


Rsquare 

Adj  Rsquare 

Root  Mean  Square  Error 

Mean  of  Response 

Observations  (or  Sum  Wgts) 

t  Test 

Assuming  equal  variances 
Difference 


Estimate 
Std  Error 
Lower  95% 
Upper  95% 


-0.35495 

0.03830 

-0.43026 

-0.27965 


UtrEqual  Variances 

Difference 


Estimate 
Std  Error 
Lower  95% 
Upper  95% 


-0.35495 

0.03830 

-0.43035 

-0.27956 


Analysis  of  Variance 


0.189252 

0.187049 

0.368335 

0.490991 

370 


t  Test 
-9.268 


DF 

368 


Prob  >  I 
<•0001 


t  Test 
-9.268 


DF 

271.786 


Prob  >  I 
<•0001 


Source 

DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Prob  >  F 

Ar  Pre/Post2  1 

11.654354 

11.6544 

85.9019 

<.0001 

Error 

368 

49.926727 

0.1357 

C.  Total 

369 

61.581081 

Means  for  Oneway  Anova 

Level 

Number 

Mean  Std  Error 

Lower  95% 

Upper  95% 

Ar  Post2 

185 

0.313514  0.02708 

0.26026 

0.36677 

Ar  Pre2 

185 

0.668468  0.02708 

0.61522 

0.72172 

Std  Error  uses  a  pooled  estimate  of  error  variance 
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Oneway  Analysis  of  Tac  2  values  By  Tac  Pre/Post2 


w 

CD 


CD 

> 

CNI 

O 

CD 


i.1t 

1- 

0.9- 

0.8- 

0.7- 

0.6-- 

0.5- 

0.4- 

0.3- 

0.2- 

0.1- 

0- 

-0.1- 


Tac  Post2 


T ac  Pre2 


Tac  Pre/Post2 


Oneway  Anova 
Summary  of  Fit 


Rsquare 

Adj  Rsquare 

Root  Mean  Square  Error 

Mean  of  Response 

Observations  (or  Sum  Wgts) 

t  Test 

Assuming  equal  variances 
Difference 


Estimate 
Std  Error 
Lower  95% 
Upper  95% 


-0.15135 

0.02270 

-0.19599 

-0.10671 


UtrEqual  Variances 

Difference 


Estimate 
Std  Error 
Lower  95% 
Upper  95% 


-0.15135 

0.02270 

-0.19599 

-0.10671 


Analysis  of  Variance 


0.107789 

0.105364 

0.218314 

0.656757 

370 


t  Test 
-6.668 


DF 

368 


Prob  >  I 
<•0001 


t  Test 
-6.668 


DF 

363, 


143 


Prob  >  I 
<•0001 


Source 

DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Prob  >  F 

Tac  Pre/Post2 

1 

2.118919 

2.11892 

44.4583 

<0001 

Error 

368 

17.539189 

0.04766 

C.  Total 

369 

19.658108 

Means  for  Oneway  Anova 

Level 

Number 

Mean  Std  Error 

Lower  95% 

Upper  95% 

Tac  Post2 

185 

0.581081  0.01605 

0.54952 

0.61264 

Tac  Pre2 

185 

0.732432  0.01605 

0.70087 

0.76400 

Std  Error  uses  a  pooled  estimate  of  error  variance 
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Oneway  Analysis  of  Mem  2  values  By  Mem  Pre/Post2 

1.1 
1 

0.9 
0.8 
S  0.7 

I  0.6 

CM  0.5 

i  0-4 

^  0.3 
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0 
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Oneway  Anova 
Summary  of  Fit 
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Rsquare 

Adj  Rsquare 

Root  Mean  Square  Error 

Mean  of  Response 

Observations  (or  Sum  Wgts) 

t  Test 

Assuming  equal  variances 
Difference 


Estimate 
Std  Error 
Lower  95% 
Upper  95% 


0.135135 

0.037994 

0.060423 

0.209847 


UtrEqual  Variances 

Difference 


Estimate 
Std  Error 
Lower  95% 
Upper  95% 


0.135135 

0.037994 

0.060407 

0.209863 


Analysis  of  Variance 


0.033234 

0.030607 

0.365413 

0.721622 

370 


t  Test 
3.557 


DF 

368 


Prob  >  I 
0.0004 


t  Test 
3.557 


DF 

345.905 


Prob  >  I 
0.0004 


Source 

DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Mem  Pre/Post2  1 

1.689189 

1.68919 

12.6506 

Error 

368 

49.137838 

0.13353 

C.  Total 

369 

50.827027 

Means  for  Oneway  Anova 

Level 

Number 

Mean  Std  Error 

Lower  95% 

Upper  95% 

Mem  Post2 

185 

0.789189  0.02687 

0.73636 

0.84202 

Mem  Pre2 

185 

0.654054  0.02687 

0.60122 

0.70688 

Std  Error  uses  a  pooled  estimate  of  error  variance 


Prob  >  F 
0.0004 
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Oneway  Analysis  of  Cog  2  values  By  Cog  Pre/Post2 
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1.1^ 
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0.5- 

0.4- 

0.3- 

0.2- 

0.1- 

0- 
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Cog  Post2  Cog  Pre2 


Cog  Pre/Post2 

Oneway  Anova 
Summary  of  Fit 


Rsquare 

Adj  Rsquare 

Root  Mean  Square  Error 

Mean  of  Response 

Observations  (or  Sum  Wgts) 

t  Test 

Assuming  equal  variances 
Difference 


Estimate 
Std  Error 
Lower  95% 
Upper  95% 


0.064865 

0.037245 

-0.00837 

0.138104 


UtrEqual  Variances 

Difference 


Estimate 
Std  Error 
Lower  95% 
Upper  95% 


0.06486 

0.03724 

-0.00840 

0.13813 


Analysis  of  Variance 


0.008175 

0.00548 

0.358207 

0.756757 

370 


t  Test 
1.742 


DF 

368 


Prob  >  I 
0.0824 


t  Test 
1.742 


DF 

336.934 


Prob  >  I 
0.0825 


Source 

DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Prob  >  F 

Cog  Pre/Post2 

1 

0.389189 

0.389189 

3.0331 

0.0824 

Error 

368 

47.218919 

0.128312 

C.  Total 

369 

47.608108 

Means  for  Oneway  Anova 

Level 

Number 

Mean  Std  Error 

Lower  95% 

Upper  95% 

Cog  Post2 

185 

0.789189  0.02634 

0.73740 

0.84098 

Cog  Pre2 

185 

0.724324  0.02634 

0.67254 

0.77611 

Std  Error  uses 

a  pooled  estimate  of  error  variance 
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Oneway  Analysis  of  Cog  Prel/2  values  By  Cog  Prel/Pre2 
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CD 
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CD 
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CD 

it  0.4- 
^  0.3- 
0.2- 
0.1- 
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Cog  Prel 


Cog  Pre2 


Cog  Pre1/Pre2 

Oneway  Anova 
Summary  of  Fit 


Rsquare 

Adj  Rsquare 

Root  Mean  Square  Error 

Mean  of  Response 

Observations  (or  Sum  Wgts) 

t  Test 

Assuming  equal  variances 
Difference 


Estimate 
Std  Error 
Lower  95% 
Upper  95% 


-0.36117 

0.04156 

-0.44288 

-0.27945 


UtrEqual  Variances 

Difference 


Estimate 
Std  Error 
Lower  95% 
Upper  95% 


-0.36117 

0.04131 

-0.44293 

-0.27940 


Analysis  of  Variance 


0.16839 

0.166161 

0.402348 

0.541333 

375 


t  Test 
-8.691 


DF 

373 


Prob  >  I 
<•0001 


t  Test 
-8.742 


DF 

316.924 


Prob  >  I 
<•0001 


Source 

DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Cog  Prel/Pre2  1 

12.226688 

12.2267 

75.5276 

Error 

373 

60.382646 

0.1619 

C.  Total 

374 

72.609333 

Means  for  Oneway  Anova 

Level 

Number 

Mean  Std  Error 

Lower  95% 

Upper  95% 

Cog  Prel 

190 

0.363158  0.02919 

0.30576 

0.42055 

Cog  Pre2 

185 

0.724324  0.02958 

0.66616 

0.78249 

Std  Error  uses  a  pooled  estimate  of  error  variance 

Prob  >  F 
<.0001 
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Appendix  B:  Screenshots  of  Text  Testing  Scenarios 


Figure  3:  Session  One  Pretest  Screenshot 

The  screen  capture  failed  to  capture  the  text,  so  the  text  is  detailed  below: 

Q:  If  your  best  friend  was  cheating  on  his  or  her  spouse,  and  you  knew  about  it, 
what  would  you  do? 

A:  ....  I  was  reading  something  the  other  day  about  infidelity.  70%  of  men  will 
have  an  affair  in  their  lifetime  and  60%  of  women . 

Q:  Hmm,  but  if  you’re  like  I  said,  if  you  had  a  friend  that  you  knew  was  cheating, 
what,  you  know,  would  you  do?  What  would  you  do? 

A:  How  close  of  a  friend? 

Q:  Um,  well,  uh,  your  best  friend.  It’s  your  best  friend. 
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A:  Well,  it  will  depend  on  what  was  going  on  in  their  relationship. . . 

Q:  Ok,  and  let’s  say  that  all  other  things  being  equal.  You  find  your  best  friend  is 
being  unfaithful,  is  having  an  affair,  urn,  what  would  be  yoiur  first  reaetion? 

A;  I  might  be  a  little  upset. 

Q:  And,  why  would  you  be  upset? 

A:  Beeause  it’s  my  best  friend 

Q:  Just  beeause  it’s  your  best  friend,  or  would  it  be  something  else? 

A;  Beeause  my  best  friend  is  involved  in  and  uneomfortable  situation . 

Q:  Uncomfortable  for  who? 

A:  For  my  best  friend. 
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Figure  4:  Session  One  Posttest  Screenshot 


Below  is  a  transcript  of  the  text: 

Q:  Um,  what  type  of  people  tend  to  rub  you  the  wrong  way? 

A:  . . .  .mm,  let’s  see  uh,  there’s  a  million  types  of  people,  uh,  umm,  let  me  think: 
controlling. 

Q:  Why? 

A:  People  that  control  me. 

Q:  Are  we  talking,  total  control  or  are  we  talking,  um,  give  me  an  example,  give 
me  a  situation  when  you  consider.. 

A:  Anyone  that  has  control  over  me. 
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Figure  5:  Session  One  Posttest  Screenshot  Page  2 


Q;  You’re  in  the  Army,  you’re  being  eontrolled  everyday 

A;  Yeah,  like  I  said  there  are  numberous  types  of  those  people  I  don’t  like 

Q;  But  then  you’re  saying  that  you  don’t  like  any  of  you  superiors. 

A:  I  just  don’t  like  people  who  eontrol  me. 

Q;  But  you  put  up  with  the  military,  you’re,  as  high  ranked  as  you  are. 

A;  Yeah,  I  know.  But  they,  like  you  said,  you  asked  me  if  they  rub  me  the  wrogn 
way,  so  I,  anybody,  I  like  to  be  in  control  and  when  somebody  has  control  over 
me,  they  rub  me  the  wrong  way. 
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Figure  6:  Session  Two  Pretest  Screenshot 


Transcript  of  text: 

Q:  What  events  do  you  remember  most  about  your  childhood,  fondly? 

A:  That’s  really  kinda  hard  to  say.  Um,  I  think  that  my  fondest  memories  of 
school.  Um,  and  growing  up  in  a  school  environment.  Those  are  my  fondest 
memories.  Um,  you  know,  I  think  highly  of  my  parents  too,  and  that’s  a  real  fond 
memory.  We  had  a  close  relationship. 
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I  Agent99 -Trainer (Quizzes) 


Figure  7:  Session  Two  Posttest  Screenshot  Page  1 


Text  transcript: 

Q:  Um,  how  would  you  describe  your  religion  and  your  religious  beliefs? 

A:  How  would  I  describe  my  religion?  I  mean  everybody  has  then  opinion  on 
religion,  uh,  it’s  kind  of  a  personal  question. 

Q:  I  realize  that.  But  how  would  you  describe  yours? 

A:  I  feel  like  I’m  as  religious  as  the  next  guy. 

Q:  Which  is? 

A:  Mm,  Religious . I  think  I’m  very  religious 


69 


»  Agent99 -Trainer (Quizzes) 


File 


Q:  Whet  is  your  religion? 

A:  Whet  is  my . what  is  that,  kind  of  a 

persond  type  question? 

Q:  Yeah. 

A:  Yeah, . you  know,  religion  is  so  broad,  I 

mean,  you  know,  religion  is  a  fui  spectrum  of 
ibfTerent  types  of  belieis. 

Q:  Right,  but  you  don't  feel  like  talking  about 
even  your  main,  spedfk  religion. 

A:  My  specific:  no.  ro. 
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Figure  8:  Session  Two  Posttest  Screenshot  Page  2 


Text  transcript: 

Q:  What  is  yoiur  religion? 

A:  What  is  my. .  .what  is  that,  kind  of  a  personal  type  question? 

Q:  Yeah. 

A:  Y eah, . . .  .you  know,  religions  is  so  broad,  I  mean,  you  know,  religion  is  a  full 
sprectrum  of  different  types  of  beliefs. 

Q:  Right,  but  you  don’t  feel  like  talking  about  even  your  main,  specific 
religion. 

A:  My  specific:  no.  no. 
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Appendix  C:  Agent99  Usability  Questions/Comments 


1.  Please  describe  any  problems  that  you  experienced  using  Agent99  Trainer. 

I  found  agent99  to  be  an  effective  training  platform;  however  I  did  not  connect  with 
the  material  being  taught  to  the  level  where  I  feel  confident  discerning  real  world 
deception.  I  believe  this  is  a  useful  training  instrument  and  would  like  to  use  it  more  and 
learn  more  about  detecting  deception. 

At  times  it  moved  to  quickly  to  really  learn  the  information.  I  am  not  a  very  fast 
reader  so  I  had  trouble  making  sure  I  read  all  of  the  words  on  the  screen  when  they  were 
there. 

The  written  portion  of  the  examples  was  too  fast  and  did  not  allow  enough  time  to 
read  them.  I  was  required  to  skim  instead  of  ACTUALLY  take  in  the  information.  This 
training  is  not  the  caliber  of  what  I  would  suggest  using  for  quality  purposes.  It  is  a  mass 
training  method  that  lacks  the  necessary  interaction  of  an  instructor.  This  program  could 
be  used  for  review  methods  but  not  as  a  primary  means  for  instruction. 

Sometimes  the  reading  screens  moved  through  the  slides  too  fast  for  me  to  read  them 
accurately. 

The  program  itself  was  not  well  written;  it  had  problems  with  synchronization  and 
robustness.  The  attention  to  details  was  low,  for  instance  there  were  over  sized  screens 
and  empty  choices.  I  would  suggest  having  someone  with  actual  programming  skills 
write  the  application. 

The  video  window  was  too  small.  The  poor  quality  of  the  example  videos  made  it 
difficult  to  tell  who  was  talking  right  away. 

I  had  to  do  the  last  set  of  questions  twice  and  then  only  when  time  expired  did 
everything  properly  continue 

The  video,  notes,  and  lecture  synchronization  were  off.  Sentences  would  end  before 
the  lecturer  was  completed  making  his  statements.  The  lecture  notes  did  not  match  the 
speaker’s  words.  Several  screens  towards  the  end  of  the  session  did  not  present  any 
information.  Received  errors. 

At  times  the  reading  was  gone  before  I  was  able  to  finish  it.  Either  build  in  a  longer 
time  to  read  it  or  make  it  so  the  person  can  advance  it  him/herself 

Some  quality  of  the  text  within  the  written  examples  was  poor  at  times  and  often  it 
moved  to  rapid  to  the  next  section. 

Using  headphones  is  irritating.  My  ears  hurt  after  the  first  20  min  thus  making  me 
less  likely  to  pay  attention  to  the  lecture.  I  removed  them  several  times  to  get  some 
feeling  back  in  them. 

The  program  didn't  initialize  on  my  first  attempt. 

No  problems,  use  of  program  ran  smooth. 

Occasional  "cannot  display  internet  explorer  page"  pop  up  messages 

Some  audio/video  stopped  short 
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At  one  point  when  using  the  trainer,  the  next  section  did  not  come  up  and  some  of 
the  fdes  needed  to  be  re-copied  from  the  CD.  Also,  a  few  of  the  videos  cut  off  before  the 
end. 

Some  of  the  video/audio  clips  of  the  lecture  cut  off  before  the  lecturer  completed 
what  he  was  saying.  The  examples  where  I  had  to  read  the  transcripts  of  conversations 
went  by  too  fast...  I  forgot  about  pausing  the  program  and  I  didn't  want  to  go  over  time. 

I  found  that  if  I  missed  something  or  wanted  to  hear  something  again  there  was  no 
way  to  go  back  or  "rewind."  Also,  on  the  reading  examples  the  slides  were  often  changed 
too  fast.  It  would  be  helpful  if  there  were  instantaneous  feedback  on  the  quizzes  so  you 
would  know  if  you  answered  correctly  or  not  and  why. 

Sometimes  the  videos  were  clipped  so  that  I  had  to  read  the  text  to  get  all  the 
information.  The  audio  was  a  bit  low  during  the  examples  and  could  be  normalized  a  bit. 
The  transcripts  were  hard  to  read  at  times. 

When  the  transcripts  of  chat  sessions  or  conversations  came  up  I  didn't  have  time  to 
read  them. 

Um...Well.  It  went  kind  of  quickly  on  some  sections  of  the  lecture.  With  the  whole 
time  thing  going  on  the  bottom,  I  uh...  felt  as  though  I  should  push  on  with  hopes  that  the 
material  I  didn't  fully  grasp  wouldn't  be  testable.  Tm  pretty  smart  [close  to  being  genius] 
but  some  sections  went  too  fast.  Tm..  umm...  not  sure  on  what  section  that  happened.  I 
umm...  think  it  was  somewhere  in  the  middle. 

The  written  examples  I  had  to  read  went  too  fast.  I  was  trying  to  examine  the  details 
of  the  written  Q&A  sessions,  but  they  jumped  to  the  next  part  before  I  was  actually  ready. 
I  purposely  read  slower  to  try  and  see  if  anything  particular  stood  out.  It  would  be  nice  if 
the  user  could  manually  change  pages  when  they  were  ready. 

I  had  some  trouble  with  the  slides  changing  before  the  speaker  was  done  talking. 
Some  of  the  text  examples  had  too  much  text  to  read  before  the  slide  changed.  It  would 
also  be  helpful  if  we  saw  some  of  the  answers  to  the  questions  given,  like  practice 
questions. 

The  text  was  distracting  since  it  did  not  always  match  what  the  person  in  the  video 
was  saying. 

One  of  the  questions  that  used  the  word  excellent  misspelled  the  word  excellent. 

There  were  times  when  I  would  have  liked  to  go  back  one  slide  to  re-play  it  but  I 
didn't  know  how. 

Volume  was  inconsistent  with  videos  /  audio. 

Audio  levels  had  to  be  adjusted  for  the  difference  in  examples  and  lecturers 

I  encountered  no  problems 

Being  that  I  was  tired,  it  was  hard  for  me  to  really  concentrate  on  what  was  going  on. 
I  believe  if  I  weren't  tired  it  would  have  been  easier  for  me  to  keep  up  with  the  lesson. 

Had  it  been  in  a  regular  classroom  type  lecture,  I  would  of  felt  a  bit  more  apt  to 
concentrate  as  to  me  a  teacher  or  instructor  gives  me  a  sense  of  "material"  importance. 
Other  than  that,  I  liked  Agent99. 

I  did  not  experience  any  technical  difficulties  with  the  Agent99  Trainer  itself; 
however,  I  did  find  it  difficult  to  follow  the  explanations  of  what  to  look  for  to  detect 
deception.  It  will  probably  be  more  effective  if  the  presenters  explain  the  different 
methods  one  at  a  time  and  then  show  visual  examples  —  video  or  audio  —  immediately 
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following  each  explanation.  I  think  this  would  help  me  to  more  easily  identify  what  to 
look  for  when  determining  if  someone  is  being  deceptive. 

I  didn't  have  time  to  read  and  completely  grasp  the  situations  before  the  screen 
automatically  changed. 

None,  I  think  I  would  like  the  ability  to  press  ahead  of  the  some  of  the  videos.  Maybe 
review  parts  of  the  videos.  And  sometimes  have  the  ability  to  skip  parts. 

This  'deception'  lesson  and  'agent  99'  test  was  too  long  to  be  sitting  in  front  of  a  pc 
with  headphones  on 

Nothing  with  the  trainer.... just  the  air-conditioning  was  SO  COLD! 

Slides  that  needed  reading  progressed  faster  than  normal  reading  speed.  Speech 
processing  speed  needs  to  slow  down 

Video  cut  short  on  the  lecturer  on  a  couple  of  occasions. 

At  the  end  of  "Effect  of  Warning  in  Information  Manipulation",  everything  stopped 
and  I  needed  to  ask  the  instructor  for  help. 

1 .  The  time  given  for  the  reading  slides  was  too  short. 

Sometimes  the  lecture  audio  would  cut  off  before  the  Power  Point  slide  changed. 

The  audio  portions  varied  in  volume  so  I  had  to  keep  making  adjustments.  Some  of  the 
sound  clips  sounded  muffled  or  had  a  little  echo  due  to  the  room  it  was  recorded  in.  No 
offense  against  people  with  cleft  pallets,  but  the  female  with  a  cleft  pallet  was  hard  to 
understand. 

The  trainer  did  not  bring  up  the  file  list  on  initial  opening.  The  different  volumes  of 
the  voices  are  very  annoying.  The  examples  at  the  end  did  not  play. 

Too  much  information  on  the  slides,  script  was  not  accurate  to  what  were  actually 
being  spoken,  and  the  video  cuts  off  at  end  of  clip  before  finishing  all  that  was  supposed 
to  be  said.  Examples  never  told  you  whether  you  were  right  or  wrong,  no  feedback  for  the 
student  using  Agent99  on  whether  he  is  actually  learning 

Eor  slow  readers  the  Q/A  slides  were  a  little  fast  and  the  audio  and  video  in  some 
parts  ended  too  abruptly. 

Some  lecture  videos  were  cut  off  before  all  info  was  passed  on 

There  were  a  couple  videos  where  the  end  got  cut  off/interrupted  and  another  video 
started. 

Topics  should  be  grouped  so  they  are  easier  to  find  if  the  user  needs  to  go  back  and 
review 

Some  of  the  audio  clips  were  "choppy"  or  cut-off  before  the  speaker  had  finished 
talking 

The  program  locked  up  after  Deception  and  Communication  Media  and  I  had  to  go 
directly  to  the  Post  test.  I  didn't  use  the  keyword  or  ask  a  question  feature  because  I 
barely  had  time  to  get  through  the  training  at  the  pace  the  system  automatically  moves. 

Some  of  the  text  did  not  match  the  audio  words  of  the  instructors.  That  can  be 
somewhat  distracting. 

Some  of  the  audio  was  cut  off  when  they  switched  sections.  The  only  other  problem 
was  that  many  of  the  things  taught  as  deception  cues  are  normally  used  by  me  and  many  I 
know  when  communicating  normally,  which  makes  it  a  bit  more  difficult  for  me. 
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The  extended  time  it  took  from  going  to  one  question  to  the  next  was  very  annoying. 

I  HATE  popups 

Not  being  able  to  replay  videos  and  audios  during  the  test  made  it  hard  sometimes 
because  I  didn't  always  hear  them  well  enough. 

The  video  would  sometimes  end  prematurely  and  go  onto  the  next  subject  without 
warning,  sometimes  cutting  off  the  end  of  the  lecture. 

I  had  no  problems. 

Some  of  the  videos  and  written  text  were  not  the  same  which  made  it  a  little  hard  to 
follow  at  times. 

The  Interface  and  the  concept  are  "on  the  right  track."  I  enjoyed  the  interface  and 
found  it  very  useful  but.  it  needs  a  few  technical  refinements.  For  example,  there  were  a 
few  small  "slow-downs"  while  the  program  moved  from  one  video  to  the  next.  Also,  a 
HELP  Utility  could  be  integrated  for  those  simple  technical  and  operational  questions 
related  to  Agent99  itself  My  congratulations  on  the  Agent99  concept,  it's  the  best  hybrid 
between  CBTs  and  the  classroom  environment  I  have  seen  so  far.  Keep  up  the  good 
work... 

If  you  have  not  yet  answered  a  question,  you  cannot  flip  between  the  question 
window  and  the  lecture. 

Sometimes  on  the  portion  of  the  lecture  that  I  had  to  read,  there  was  not  enough  time 
for  me  to  read  the  slide  before  it  went  to  the  next  slide. 

The  GUI  is  just  fine 

Some  pauses  in  the  video. 

I  thought  that  the  video  screen  should  be  larger,  and  the  examples  should  be  given 
head  on  instead  of  profile  shots  of  the  individuals  interviewed.  I  also  believe  that  the  text 
examples  should  be  scroll  based  instead  of  timed. 

Some  of  the  lectures  would  get  cut  off  before  the  speaker  had  finished.  Additionally, 
the  script  (below  the  video  display)  showed  more  verbiage  than  what  was  mentioned  in 
the  video. 

I  attempted  to  use  your  Keyword  search  on  one  occasion  with  no  success.  It  could  be 
because  I  attempted  during  the  quiz.  Also,  it  would  be  beneficial  to  have  a  video/audio 
tracker  to  allow  for  you  to  go  back  to  the  beginning  of  a  lecture. 

The  issues  that  I  have  are  not  "problems",  but  mainly  suggestions.  The  script  below 
the  speakers  did  not  always  follow  what  was  being  said  (or  visa  versa)  and  I  found  myself 
concentrating  more  on  what  wasn't  being  said  and  wondering  "why"  rather  than  following 
the  lecture. 

I  wasn’t  prepared  for  the  variety  of  questions;  text,  audio,  video.  I  also  feel  that  the 
text  questions  move  to  fast  to  be  read. 

On  occasion,  I  lost  interest  due  to  over  explanation. 

Some  of  the  video  clips  moved  too  fast.  Way  too  much  going  on  the  desktop,  poor 
video  quality  on  some  of  the  clips 

I  did  not  experience  any  problems.  Overall  it's  a  good  program,  but  I  feel  the 
information  was  extensive  in  nature. 

Occasionally  the  voice  cut  out,  but  quickly  returned.  At  one  point  the  video  froze 
and  I  required  technical  assistance. 


74 


If  I  selected  a  different  topic,  the  slide  screen  in  the  bottom  right  corner  did  not 
always  follow.  Also,  there  were  two  video  examples  that  did  not  function... they  were  the 
two  before  the  short  latency  examples. 

a  lot  of  information;  some  of  the  lectures  went  by  too  quickly,  could  have  been  more 
obvious  exactly  what  the  actors  did  that  gave  away  they  were  lying  &  what  the  actors  did 
that  came  to  the  conclusion  they  were  truthful 
Did  not  encounter  any  problems. 

Some  of  the  video  was  hard  to  see  and  some  of  the  audio  skipped  when  the 
instructors  were  lecturing.  Overall  the  material  wasn't  bad.  I  just  had  to  concentrate 
harder  in  some  areas  than  I  did  in  others. 

No  problems,  but  I  assume  that  there  are  certain  periods  when  we  are  allowed  to  go 
back  and  look  for  information.. 

We  were  instructed  to  feel  free  to  skip  around  through  the  slides  as  we  wished.  After 
clicking  on  the  summary  slide  I  was  immediately  prompted  to  take  the  post-test.  I  was 
not  ready  to  take  the  post- test  and  didn't  want  to.  There  was  no  option  to  stop  the  test  or 
to  "backup". 

It  was  a  little  difficult  when  the  text  didn't  match  up  with  what  the  person  was 
saying.  Especially  if  an  entire  paragraph  was  skipped. 

Suggestion:  provide  whether  examples  are  true  are  false  to  give  trainee  a  better  idea 
of  example. 

Varying  of  the  loudness  in  the  slides 
The  written  transcripts  went  too  fast 

The  example  video  quality  was  bad  and  the  audio  wasn't  always  understandable. 

Cut  off  the  speaker  at  the  end  of  segments 

The  lag  time  for  the  pop-up  questions  was  a  bit  annoying 

It  was  strange  to  have  the  instructor  audio  cut  out  before  it  was  done  with  a  sentence 
and  sometimes  questions  popped  up  when  a  video  was  going  so  I  had  to  pick  a  choice  so 
I  could  concentrate  on  the  video,  but  other  than  that  it  was  an  excellent  program 

As  a  computer  science  major,  I  found  that  is  was  difficult  getting  started  up.  I  am 
used  to  filling  out  forms  online  and  is  was  cumbersome  at  the  beginning 
Sometimes  the  audio  would  cut  off  during  the  presentation 
No  real  problems.  Just  noticed  that  the  video  and  the  slides  were  a  little  off  sync. 

The  next  slide  would  sometimes  cut  off  the  video  that  was  playing  before  it  was  actually 
done. 

-Sometimes  the  questions  were  confusing  from  a  grammar  standpoint. 

Fatal  error.  A  progress  check  question  was  on  the  screen  and  the  system  asked  me  if 
I  wanted  to  take  the  final  test  now  because  I  was  running  out  of  time.  I  clicked  yes  and 
then  tried  to  answer  the  progress  check  question.  System  locked  and  showed  the  fatal 
error  message.  Instructor  had  to  restart  program. 

Received  error  while  using  the  program.  Also,  when  I  tried  using  the  keyword 
search,  it  tried  to  install  some  items  from  office.  Both  of  these  interruptions  took  my 
focus  away  from  the  training. 

Error  message,  had  to  correct  and  log  back  in. 

Had  a  run  error  during  use 
Had  a  runtime  error. 
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The  text  only  examples  were  not  on  the  sereen  long  enough  for  me  to  eompletely 
read  the  seenarios. 

The  video  would  cut  off  the  last  sentence,  and  the  slides  would  move  to  fast  when  I 
was  trying  to  read  them.  You  should  be  prompted  to  move  forward  to  the  next  slide. 

Was  unsure  when  "Keyword"  or  "Ask  a  Question"  section  were  available,  it  would 
not  allow  me  to  use  it  while  some  videos  were  playing 

audio  was  skipping  at  times 

Some  of  the  text-based  examples  went  too  fast.  I  prefer  to  re-read  the  text  to  make 
sure  I  understand  exactly  what  was  being  written.  I.  therefore,  may  have  missed 
important  parts  of  the  text  and  was  unable  to  answer  the  question  of  deception  accurately. 

The  embedded  audio  volume  was  a  little  high  at  times.  Some  of  the  video  examples 
where  a  little  too  grainy  so  looking  for  physical  clues  was  difficult  at  times. 

At  one  point,  a  question  popped  up  before  the  lecture  had  stopped.  I  missed  what  the 
lecture  had  to  say  while  I  was  answering  the  question  and  the  program  would  not  allow 
me  to  pause  the  lecture. 

Sometimes  the  video  portion  would  advance  before  it  reached  the  end  of  what  was 
included  in  the  caption  portion. 

Program  would  jump  to  next  topic  before  completing  the  last. 

The  video  feed  stopped  working  during  one  of  the  lectures.  We  paused  the  session 
and  it  started  working  again. 

One  run-time  error  caused  the  application  to  close  during  one  point  of  the  program. 
Agent99  is  the  best  CBT  format  I  have  ever  used.  I  hope  all  of  them  start  working  like 
this;  it  beats  death  by  power  point  anytime. 

-  Pop  Questions  would  appear  a  second  prior  to  the  video  clip  actually  finish. 

The  timing  was  a  little  off  with  the  questions  and  the  video.  The  question  would  pop 
up  before  the  instructor  was  finished. 

Experienced  a  run  time  error  during  the  program. 

In  one  section  I  was  shown  several  clips  and  asked  to  make  a  determination  of  truth 
or  deception.  After  the  program  said  something  like  2  and  4  were  truthful  (or  dishonest). 
Anyway,  I  had  seen  many  clips  by  that  point  and  they  were  jumbled  in  my  head.  End 
result  was  I  did  not  know  if  I  answered  correctly  or  not.  No  positive  reinforcement. 

Could  not  utilize  function  to  "go  back"  or  listen  to  lecture  or  example  audio  &  video 
again  if  one  existed. 

2,  What  do  you  like  about  Agent99  Trainer? 

How  it  easily  presents  information;  audio  and  visual  are  tied  together  for  interactive 
training  which  is  entertaining. 

Video  examples. 

I  think  it  is  a  good  idea  to  be  able  to  teach  classes  by  video  like  this.  The  question 
boxes  worked  well.  I  like  being  able  to  integrate  user  input  quizzes  and  the  lecture 
information  so  that  there  are  occasions  where  the  user  can  cement  their  learning  in  the 
midst  of  the  lecture. 

The  video  examples.  They  showed  the  characteristics  described  right  before. 

It  was  easy  to  use  and  continued  without  me  having  to  do  anything. 
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Nothing 

I  like  having  the  ability  to  pause  the  lectures  and  other  parts  at  any  time. 

It  gave  an  interesting  approach  to  the  subject  matter. 

Easy  to  use 

I  think  it  has  potential,  but  the  format  as  it  stands  now  is  not  impressive  to  me.  There 
is  too  much  information  on  the  screen.  Either  provide  the  video  lecture  with  overview 
slides  or  the  written  presentation,  but  not  both.  Get  rid  of  the  upper  right  box.  Serves  no 
purpose. 

I  liked  that  the  possibility  existed  to  move  at  your  own  pace. 

I  believe  this  is  the  first  attempt  at  deception  training  that  I  have  received. 

Good  video 

It's  a  nice  change  from  the  routine  classroom. 

Pause  feature 

Examples.,  and  pre  and  post  test 

Video  &  audio  samples 

Easy 

Eooks  ok 

The  video  examples 

It's  very  similar  to  a  classroom  lecture  w/  PowerPoint.  Would  be  much  more  useful 
for  online  learning  than  just  video  lectures  and  emailing  between  students/instructor. 
Allows  students  to  learn  whichever  way  fits  them  best,  lecture,  reading,  or  examples. 

Well  laid  out  with  and  easy  to  use. 

Some  of  the  examples  were  very  helpful...  it's  nice  to  have  a  transcript  of  what's 
being  said. 

Ease  of  use;  concurrent  video  with  text  transcript  and  lecture  slides 

The  text  and  video/audio  together 

It  is  a  good  way  for  individuals  to  have  training. 

Interactive  was  a  nice  feature.  The  scenarios  were  pretty  true  to  life.  More  examples 
(video,  voice,  and  written)  would  be  nice. 

It  is  user  friendly  and  the  quality  was  good.  The  examples  given  were  also  good  and 
well  encompassing 

Convenient  for  fitting  training  into  busy  schedules. 

The  simultaneous  slide  and  video/audio  presentations.  They  aided  the  learning 
process,  specifically  in  visualizing  the  examples  and  understanding  the  explained 
concepts. 

The  way  it  has  all  of  the  pains  working  together 

The  ability  to  hit  pause  so  I  could  read  the  text  at  the  bottom  left  screen.  I  was  able 
to  re-read  and  review  a  couple  of  things  that  I  might  not  have  heard  correctly  from  the 
audio. 

The  use  of  PowerPoint  along  with  the  lecture 

GUI  environment 

Slides,  lecture,  and  transcript  all  at  once,  this  allows  me  to  look  away  from  the 
lecturer  and  view  the  main  points  on  the  slide.  It  also  allows  me  to  read  a  part  of  the 
lecture  if  I  did  not  understand  what  the  lecturer  said. 

Very  interactive  training  tool 
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It  had  a  lot  of  info  at  my  disposal  and  it  integrated  the  different  learning  medias. 

I  like  its  ease  of  use  and  the  eombination  of  audio  and  video  presentation  along  with 
slide  show  and  text.  I  think  it  is  a  good  method  of  providing  a  leeture  via  a  simple 
applieation  package.  I  would  recommend;  however,  to  add  a  playback  button  to  repeat 
any  information  that  may  be  overlooked  by  the  user. 

Having  the  3  methods  going  on  at  once  was  more  effective.  I  didn't  use  it  but  I  liked 
the  ability  to  go  to  different  sections  at  any  time  by  using  the  top  right  screen. 

I  felt  that  it  was  a  very  good  implementation  and  GUI  for  learning.  The  system  was 
well  integrated  with  the  video  and  lesson  aspects. 

Integration  of  learning  material  and  multimedia 

Its  interactiveness 

Nothing 

Well  integrated. 

Good  combination  of  video  and  audio  examples  with  lecture.  Showing/testing 
examples  in  the  beginning,  lecturing,  and  then  re-testing  is  a  good  learning  method  and  a 
good  way  for  the  developers  to  determine  the  effectiveness  of  the  tool. 

Extremely  easy  to  use.  I  liked  the  video  clips.  The  examples  were  also  very  helpful. 
Simplicity.  I  liked  that  I  could  read  as  well  as  watch  the  instructor.  It  was  a  great 
use  of  multimedia  and  I  feel  like  I've  learned  more  about  deception.  Everyone  should  go 
through  this  training. 

Both  Audio  and  Visual  and  with  notes 
The  Media  player  and  the  examples 

Something  I  like  about  it  I  also  don't  like.  The  different  layouts  (video,  audio,  and 
slides)  make  it  easier  for  different  styles  of  learners.  On  the  other  hand,  I  found  myself 
trying  to  look  at  two  different  things  at  once  and  lose  the  audio. 

Eike  interactive  approach,  attempt  to  incorporate  multiple  learning  styles  into  one 
system,  real  examples  of  deception  everyday 
Simple  to  understand,  easy  to  follow  along 
-Smooth  integration  of  various  media  types 
The  integration  of  the  all  media  forms 

Video  audio  and  text  a  good  mix  although  when  reading  along  with  video  and  text 
did  not  match  exactly  I  spent  more  time  finding  where  to  pick  it  back  up  and  less  time 
listening  or  understanding  the  intended  message. 

The  interactive  quizzes 
Very  simple 

1)  Video  and  audio  examples.  2)  Teaching  with  video,  text,  and  PowerPoint 
simultaneously.  Eef  s  me  quickly  review  points  on  PowerPoint,  listen  to  instructor,  and 
review  what  he  said  when  I  missed  it.  3)  Software  is  easy  to  look  at.  Eye  friendly.  4) 
Control  over  volume  and  pause  due  to  distractions. 

I  liked  that  it  used  not  only  audio  and  video,  but  that  it  integrated  it  with  slides  and 
transcripts. 

Redundancy  of  the  material  in  each  section 
Integration  of  multimedia 
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I  like  the  ineorporation  of  Topie,  Notes,  Slides  and  Video/ Audio  all  at  onee  -  being 
able  to  foeus  you  attention  to  the  notes  and  read  along  with  the  speaker  or  look  at  the 
slides  to  get  a  good  idea  of  where  the  speaker  may  be  going  with  the  topie  is  very  useful. 

It's  very  easy  to  use. 

I  like  the  visuals  with  the  examples.  It  helps  everything  sink  in. 

Nothing 

It  has  quiek  response  times  to  ehoiees  the  user  makes 

It  foeuses  on  all  areas  of  learning.  It  allows  one  to  backtrack  and  see  sections  again. 

Ease  of  use,  integration  of  video  and  slides,  good  outline  and  plan 

I  think  it  had  some  good  information. 

Self-paced 

Lots  of  visual  information  that  I  can  control  while  learning. 

The  synchronized  briefing  with  the  slides  and  text...  The  ability  to  jump  to  any 
portion  of  the  briefing  on  demand. 

It's  very  interactive 

The  Interface  concept  itself...  It  is  the  best  hybrid  between  CBTs  and  classroom 
teaching  I  have  ever  seen.  It  would  be  an  extremely  useful  tool  for  military  and 
commercial  teaching  applications. 

The  4  windows,  the  video  with  the  text  underneath.  I  like  the  arrangement  of 
information. 

The  combination  of  way  to  learn  the  subject 

Easy  to  use 

It  is  simple  to  use 

Video  &  audio  use 

All  windows  are  open  and  if  you  want  to  interrupt  the  procedure  or  go  back,  you 
have  that  option. 

It  helped  me  to  be  a  part  of  the  classroom  setting  by: 

I  like  the  videos  and  the  ability  to  maneuver  within  the  program. 

The  integration  of  all  media  was  very  nice.  I  do  not  use  chat  rooms  on  the  internet, 
and  I  typically  find  simple  Power  Point  slides  not  detailed  enough.  Most  CBT  courses 
are  taught  in  that  chat  room  format  or  with  simple  slides.  The  lecture  and  script  were 
great  learning  tools. 

Having  both  the  audio  lectures  with  the  slides  and  some  examples  all  coinciding  is  a 
great  idea.  The  quality  of  the  audio  video  is  more  than  satisfactory 

It’s  attractive  too  all  forms  of  learners,  (visual,  verbal,  etc...) 

Examples  of  what  you  are  trying  to  teach  are  good 

Video  feeds  and  visual  examples 

Gave  some  specific  cues  to  look  for  in  regards  to  detecting  deception. 

The  ability  to  review  topics  multiple  times  if  I  have  any  questions. 

The  audio  and  video  which  went  with  the  slides.  Also,  the  video  examples. 

I  liked  the  video  examples.  I  thought  they  were  very  useful.  I  liked  them  better  than 
the  audio  or  the  read-along  examples. 

I  like  the  idea  of  receiving  deception  training. 

The  use  of  video  examples  that  followed  right  along  with  the  lecture. 

I  could  go  pretty  much  at  my  own  pace  versus  sitting  in  a  room  listening  to  a  lecture 
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I  liked  the  video  along  with  the  transeript...  It's  much  better  than  audio  and  slides,  or 
just  the  slides  alone.... 

Multiple  mediums  of  instruction  should  benefit  people  who  learn  in  different  ways. 
Unfortunately,  you  force  all  methods  on  the  user  at  once.  If  I  have  already  read  the 
transcript  of  a  lecture,  why  do  I  need  to  listen  to  somebody  else  read  the  rest  of  it?  I  learn 
better  from  reading  anyway,  if  s  really  just  wasting  my  time  at  that  point. 

It  runs  straight  through,  but  there  is  the  option  of  going  back  to  any  section  in  the 
program 

Multiple  learning  styles  supplied 

The  wealth  of  information  available,  i.e...  one  can  navigate  through,  read,  listen, 
watch,  and  see  summary  slides  simultaneously. 

I  Liked  the  use  of  power  point  as  well  as  the  integration  of  audio  and  video  in 
agent99. 

Simple  vb  interface 
Audio  examples 

The  instructors  were  knowledgeable  and  clear  spoken.  The  video  quality  for  the 
instructor  videos  was  quite  good. 

Video  with  slides  and  text 
Examples 

Uses  visual  and  two  forms  of  written  to  aid  in  learning 

The  visual  instructor  and  being  able  to  listen  while  reading  or  looking  at  the  slides. 
The  integrated  approach  of  combining  all  different  types  of  learning  techniques 
The  different  mediums  to  learn  information 
If  it  had  more  real  examples. 

User-friendly 

-I  like  that  you  can  get  your  information  from  each  type  of  medium,  it  is  good  for 
people  who  learn  differently. 

Gives  you  information  is  multi  formats  at  the  same  time. 

Multiple  displays  at  one  time:  slides,  video,  transcript,  and  outline 
Real  examples 

Easy  to  use.  I  liked  having  text  that  accompanied  the  video  and  audio,  as  the  audio 
was  sometimes  difficult  to  understand. 

Ease  of  use.  Good  examples. 

It  was  easy  to  use 
Easy  to  use. 

It's  use  of  the  various  learning  styles  (auditory,  visual,  and  tactile)  to  keep  me 
involved.  I  liked  the  pop  up  questions  most. 

Nice  interactivity 
I  like  the  video/  slide  integration. 

Good  examples  and  videos 

The  audio  and  visual  examples  along  with  text  and  the  caption  as  instructors  speak  - 1 
just  wished  it  didn't  skip  audio  every  now  and  then 
Self-Paced. 

Integration  of  audio  and  text.  The  example  videos  were  also  pretty  great. 

Integration  between  the  text/video/PowerPoint  slides 
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I  like  the  faet  that  you  get  audio,  video  and  text  instruction  all  in  one  go.  The 
instruction  coupled  with  the  questions  that  popped  up  at  times  and  the  pre/post  tests  made 
the  learning  experience  more  dynamic  and  easier  to  comprehend. 

Information  presentation.  Because  of  the  various  different  ways  you  were  able  to  see 
the  information  and  review  it;  it  allowed  you  to  learn  at  your  own  pace. 

The  ability  to  see  a  slide,  hear  how  one  talks  through  it,  but  also  see  what  the 
instructor  said  -  both  slide  and  textbook-like  information  is  nice 

I  am  a  visual  learner,  and  the  video  with  the  examples  helped  me  understand  the 
information  better. 

It  was  easy  to  use  and  very  informative. 

The  video,  slide  and  notes/commentary  combine  to  give  you  the  visual  as  well  as 
audio  aspects  that  reinforce  the  learning  material  being  presented  very  effectively. 

Well  thought  out,  audio  and  visual  stimulation  (text  and  video)  is  a  great  idea.  Not 
sure  about  the  "search  function"  perhaps  it  is  not  applicable  to  this  particular  training 
exercise 

I  like  the  ability  to  see  the  text  speech  and  the  slides  as  well  as  the  instructor,  all  at 
the  same  time. 

The  format  (video  &  audio),  with  the  instructors  talking  and  the  text  displayed. 

Slides  provided  a  good  synopsis  of  the  material  covered.  The  table  of  contents  made 
searching  other  topics  easily  accessible. 

I  liked  the  multiple  panes  with  slides,  video  and  text.  If  I  got  tired  of  looking  at  the 
speaker  I  could  read  the  text  or  refer  to  the  slides.  A  good  way  to  keep  students  attention. 

Interactive;  gives  theory  supported  by  examples 


3.  What  do  you  dislike  about  Agent99  Trainer? 

I  left  not  feeling  confident 

No  teacher/student  interaction.  Some  of  the  examples  did  not  necessarily  clearly 
illustrate  what  was  trying  to  be  expressed. 

I  don't  like  the  fact  that  there  was  no  feedback  on  your  answers.  You  answer  the 
quizzes  not  knowing  if  any  of  your  answers  are  right. 

It  is  too  vague  and  the  written  examples  are  too  fast. 

It  was  kind  of  boring  and  at  parts  I  lost  focus  and  when  the  questions  came  up  at  the 
end  I  had  trouble  remembering  what  was  said  earlier.  I  was  unsure  how  to  go  back  so  I 
couldn't  relearn  what  I  had  missed. 

I  just  explained  that  the  program  is  poorly  written. 

Not  having  the  ability  to  go  back  a  few  seconds  in  the  videos  to  catch  something  I 
may  have  missed. 

Just  going  through  things  twice 

I  would  have  gotten  more  out  of  the  training  if  after  some  the  examples  we  went 
back  and  showed  where  the  deception  occurred. 

Size  of  the  window. 

Too  small  and  I  prefer  lecture  in  person  versus  a  PC 
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More  flexibility  allowing  a  user  to  review  a  particular  video  again  would  be  nice. 

No  written  material  (paper)  that  identities  key  concepts 

Looking  at  a  screen  for  that  amount  of  time 

Nothing  that  I  really  disliked.  I  believe  it’s  quality  training 

Too  much  info 

Nothing 

It  was  made  using  visual  basic.  Looks  clunky 

Video  editing  was  rather  distracting;  audio  split  into  stereo  unnecessarily  and  was 
somewhat  distracting;  time  to  read  slides  in  realplayer  was  not  sufficient... often  changed 
before  finished  reading  the  slide;  video/audio  cut  out  before  speech  was  finished  in  a  few 
sections;  lack  of  introduction  before  diving  into  the  details  of  the  subject  made  it  difficult 
to  understand  what  it  was  I  was  doing. 

Some  slides  where  reading  was  required  did  not  allow  enough  time  for  me  personally 

I  dislike  the  fact  that  the  recorded  instructors  are  basically  reading  off  the  notes. 

Can't  think  of  anything. 

No  way  to  go  back  and  listen  to/see  something  again.  No  way  to  ask  questions  or 
interact. 

Some  audio  did  not  match  transcripts;  final  seconds  of  some  audio  segments  were 
clipped 

The  video  was  clipped  at  points  and  the  audio  should  be  normalized 

Needs  more  controls. 

Not  being  able  to  control  the  speed  of  activities  requires  scrutinization  of  written 
examples.  Video  footage  could  be  better. 

The  text  window. 

Some  of  the  slides  moved  too  fast  and  didn't  allow  the  user  to  finish  reading 
everything.  The  controls  on  the  media  player  should  be  left  in  tact  so  that  in  cases  where 
readers  are  slower  than  the  media  is  set  to  then  they  can  rewind  or  replay  without  having 
to  start  that  portion  over. 

Nothing 

There  was  no  immediate  feedback.  I  saw  numerous  examples  of  'possible'  deception, 
made  my  best  guess,  but  then  didn't  find  out  if  I  was  right  or  not  until  the  end  of  that 
section.  Hell,  I  can't  remember  what  example  one  was  compared  to  example  5  or  6.  I 
found  that  annoying.  I  didn't  know  if  I  was  picking  it  up  well  or  not  because  it  didn't  tell 
me  until  the  end:  "example  X  &  Y  were  deceptive,  the  rest  were  truthful". 

Went  to  fast 

Redundancy.  Some  video  clips  were  cut  short. 

This  particular  lesson  was  too  long.  Thirty  minutes  or  less  should  be  the  target. 

Sometimes  it  went  a  little  to  fast 

I  personally  do  not  like  learning  from  computers.  I  just  feel  that  my  eyes  get  really 
tired  and  it  makes  me  disinterested. 

Again,  I  didn't  like  not  being  able  to  playback  something  I  missed  the  lecturer  saying. 
As  far  as  the  information  itself,  I  did  not  like  the  lack  of  verbal  examples  from  the 
lecturers  and  the  lack  of  visual  examples  mixed  within  the  lecturers  lengthy  explanations. 

Certain  topics  like  the  one  we  went  over  require  discussion  and  personal  explanation 
to  sometimes  see  complex  details  that  aren't  very  obvious  the  first  time  you  look  at  it 
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I  did  not  have  the  ability  to  skip  ahead.  Or  did  not  know  I  eould  skip  ahead,  review 
parts  or  serub  through  the  video. 

Impersonal 

At  times  the  example  slides  with  seript  when  too  quickly  from  one  slide  to  the  next. 

Nothing 

Very  long... 

Some  of  the  sound  quality. 

Headsets 

The  only  thing  I  didn't  like  is  that  the  reading  examples  needed  more  time  on  the 
screen.  I  have  a  reading  problem  and  needed  more  time  to  read  and  comprehend  the 
example. 

Size  of  video  image  made  it  pointless  at  times 

Audio  volume  inconsistent  and  for  the  short  audio  clips  it  was  hard  to  tell  if  someone 
is  being  deceptive  with  just  a  sound  blurp. 

I  would've  liked  to  been  able  to  see  if  my  deception  detections  were  correct. 

Slides  are  grainy 

1)  History  of  research  at  end  is  interesting.  But  if  Agent99's  purpose  is  to  teach  me 
better  deception  detection  skills,  then  history  of  research  is  irrelevant.  Also,  Agent99 
focused  on  face  to  face  interaction.  Very  shallow  review  of  deception  detection  in  other 
mediums  like  email,  video  conferencing,  etc.  I  would  like  to  learn  about  these  too. 

Some  of  the  video  examples  were  hard  to  see  and  a  little  difficult  to  understand. 

I  think  there  should  be  instantaneous  feedback  for  some  of  the  videos.  A  voiceover 
explaining  specific  examples  of  indicators  of  deception  as  they  occur  in  the  clips  would 
be  helpful. 

Not  knowing  how  many  questions  there  were 

Not  a  huge  problem,  but  I  just  let  the  program  take  its  course  -  it  would  have  been 
more  helpful  if  the  Topic  section  had  updated  as  I  automatically  switched  topics  -  so  that 
I  could  know  my  position  within  the  available  material  as  well  as  a  reminder  of  which 
subheading  the  current  lecture  fell  under.  Instead,  it  stayed  on  the  "Introduction"  for  the 
course. 

It  doesn't  keep  my  attention  well  since  the  screen  with  the  instructors  is  so  small. 

The  people  speaking  were  kind  of  monotone.  Nothing  really  wrong  with  that 
because  it's  a  learning  environment  but  it  made  me  sleepy. 

Too  slow 

It  seems  like  technology  that  is  trying  to  replace  the  classroom.  I  think  the 
interactive  classroom  is  still  the  most  effective  means  of  communicating  information. 

I  cannot  rewind  the  video 

Some  of  the  examples  are  a  little  iffy,  could  go  either  way,  and  much  depends  on  the 
person  who  is  speaking. 

Latency  and  the  popup  questions 

Not  being  able  to  replay  questions 

I  couldn't  advance  through  before  completing  the  pretest 

Nothing. 

I  didn't  have  any  use  for  the  Search  during  this  training  so  have  no  opinion  on  it,  but 
everything  else  seemed  to  work  fine. 
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No  feedback  was  given  for  the  pretest  at  all.  Also  many  of  the  tests  were  ambiguous. 
It  is  hard  to  detect  deception  when  you  have  no  context  for  a  video  or  audio  clip 

The  lack  of  a  HELP  Utility  associate  with  the  program  itself  For  example,  I  find  the 
"Paper  Clip"  Help  character  in  Microsoft  Office  very  inconvenient  but  when  I  have  a 
question  about  how  to  do  or  fix  something  with  the  program  or  my  work  it  always  has 
"tons  of  answers"... 

Long 

Becomes  boring 

This  last  test 

Your  tests  were  geared  towards  me  detecting  when  a  person  was  deceiving.  I  don't 
think  enough  time  was  spent  on  learning  the  detectors.  I  felt  let  down  in  that  area,  as 
though  I  was  not  given  enough  tools  to  make  an  educated  guess 

Kind  of  long 

The  video  window  seemed  small.  Maybe  a  higher  quality  would  make  the  use  better. 

Overall,  I  had  no  problems  with  the  trainer.  This  is  an  excellent  tool  for  training. 

Too  much  information....  Also,  would  like  to  have  to  known  if  I  answered  questions 
correctly  during  the  quiz.  I  feel  Tm  smarter  now,  but  I  don't  really  know. 

Again,  issues  with  the  topic  selector  not  highlighting  the  current  topic,  and  the  script 
not  being  word-for-word  with  the  instructors. 

The  biggest  drawback  is  that  the  trainer  doesn’t  seem  reusable.  When  I  say  not 
reusable  I  refer  to  the  lack  of  interaction.  I  would  not  want  to  sit  though  all  of  these 
slides. 

Some  of  the  modules  offered  an  over  explanation. 

I  have  this  overwhelming  feeling  of  distrust  towards  everybody  right  now.  Going  to 
throw  up  way  too  many  false  alarms. 

Too  lengthy,  not  enough  time  looking  at  one  deception  attribute 

Took  to  long  to  get  the  point  across. 

I  have  no  complaints  when  comparing  this  learning  technique  to  a  standard  lecture. 

The  video  quality  was  a  little  poor. 

I  don't  like  that  you  cannot  go  back  on  your  test  questions  or  these  answer  responses. 
It  is  also  difficult  to  just  go  back  a  little  bit  on  your  slides  and  examples  when  I  can't 
read-along  as  quickly  as  the  slides  move. ..only  happened  a  few  times. 

Some  of  the  lecture  was  too  fast.  Could  have  been  more  explicit  during  the  acting 
scenes. 

The  fact  that  I  was  not  given  any  feed  back  on  either  the  pre-test  or  the  post-test.  I 
would  have  liked  to  have  been  told  if  my  answers  were  correct,  and  if  they  were  wrong 
what  the  correct  answer  was. 

It  didn't  display  what  the  average  answer  was  or  what  the  textbook  answer  should  be 

While  there  are  good  things  about  having  all  of  these  features  combined  into  one 
application,  it  is  bad  to  have  to  use  it  with  no  breaks.  It  can  be  difficult  and  tiresome 
sitting  in  one  place  staring  at  the  screen. 

You  force  all  methods  on  the  user  at  once.  If  I  have  already  read  the  transcript  of  a 
lecture,  why  do  I  need  to  listen  to  somebody  else  read  the  rest  of  it?  I  learn  better  from 
reading  anyway;  it's  really  just  wasting  my  time  at  that  point. 

The  UI  isn't  visually  appealing.  It's  bland. 
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Some  of  the  audio  elips  were  hard  to  understand.  Also,  when  a  reading  question  was 
asked,  the  trainer  moved  on  too  quiekly. 

The  transeript  portion  of  agent  99  did  not  eompletely  mateh  with  what  the  audio  was 
saying.  There  were  whole  sentenees  missing  and  that  eaused  me  to  lose  my  plaee/train  of 
thought  a  eouple  of  times. 

The  video  slides  and  the  keyword...  too  mueh...  It  would  be  easier  to  visually  aeeept 
if  less  erowded 

The  verbal  reading  of  the  instructors,  at  times  it  became  a  distraction 

Lack  of  interaction.  Still  feel  unsure  about  deceivers  unless  is  very  obvious  fear  or 
equivocating. 

Too  slow 

Poor  video  quality  in  examples 

There  was  a  loud  beep  after  answering  some  of  the  questions 

Video  guys  talk  to  slow,  should  be  an  option  to  watch  the  video  or  to  end  it  if  you 
finish  reading  the  material  early 

Sometimes  it  went  too  fast  for  me  to  really  comprehend  the  subject,  but  I  could 
always  go  back  if  I  needed  to 

Feeling  like  I  was  timed 

Nothing 

It  seems  impersonal  and  a  bit  "campy"  sometimes. 

Video  a  little  choppy  in  places,  test  questions  are  slow  to  change  once  you  have 
answered  the  question.  It  would  be  nice  if  the  list  of  sections  showed  which  section  you 
were  on  so  you  know  how  far  you  had  gotten  in  the  training. 

Nothing 

I  would  have  liked  to  know  what  section  I  was  in.  I  was  not  always  sure  (by  looking 
at  the  selectable  section  area)  where  I  was.  I  would  have  liked  to  have  a  "Next"  button 
instead  of  it  automatically  advancing.  Sometime  I  wanted  to  reread  a  section. 

That  I  had  to  use  it 

Information  not  being  displayed  in  a  timely  manner. 

Nothing 

The  limited  time  for  the  text  examples. 

Would  like  a  lot  more  examples  and  immediate  feedback  on  whether  or  not  my 
ability  to  detect  deception  was  accurate. 

The  overview  top  right  box  should  be  a  drop  down  menu.  It  takes  up  too  much  of  the 
screen. 

No  breaks  between  sections;  made  navigating  at  my  own  pace  un-easy 

It's  moving  so  fast  and  it  is  timed  -  I'd  like  to  take  my  time  with  it,  pause  it  when  I 
feel  like  it,  and  think  about  the  info  that  was  given  to  me.  also  being  able  to  replay  the 
info  given  to  me  -  it  seems  it  was  going  too  quick  for  me. 

Nothing. 

Sometimes  the  lecture  portion  did  not  match  up  with  the  text  in  the  bottom  left 
screen.  I  found  that  to  be  distracting. 

Nothing  really. 

I  would  have  preferred  to  not  have  had  a  timer  on  and  also  to  have  had  an 
opportunity  to  review  my  previous  responses  and  what  the  correct  answers  were. 
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That  you  have  to  view  the  instruetions  on  how  to  use  it  every  time  it  starts  up. 
Nothing 

Faet  that  I  was  unable  to  eomplete  the  entire  session  beeause  of  the  time  eonstraints. 
The  right-hand  window  that  shows  the  eourse  outline  needs  to  highlight  the  slide  / 
presentation  area  being  presented  by  the  video  and  slide  areas. 

App  Crash 
1 .  Modify  the  UI: 

The  video  or  audio  feed  cutting  off  before  the  lecture  appeared  finished. 

Several  examples  used  a  transcript  version  we  were  supposed  to  read.  The 
information  automatically  advanced  and  several  times  I  was  unable  to  read  all  of  it  (I  am 
a  slow  reader  sometimes).  I  surmised  this  might  be  intentional  to  avoid  giving  an 
inordinate  amount  of  time  to  analyze  the  statements. 

Would  like  a  "go  back"  or  repeat  function  on  certain  slides;  otherwise,  Tm  very 
satisfied  with  the  trainer. 

4,  How  can  Agent99  Trainer  be  improved  to  better  help  you  learn? 

Better  define  when  people  are  lying 
Larger  video  size. 

I  think  feedback  on  the  questions  that  popped  up  could  be  very  useful  in  most 
situations.  More  freedom  to  slow  it  down  or  speed  it  up  might  help. 

Decrease  the  speed  of  the  written  example  scrolling.  Please  also  provide  more 
descriptive  terminology  concerning  the  instruction 

I  am  not  sure,  a  little  more  interaction  would  help.  For  example,  when  showing 
examples  allow  for  a  function  for  us  to  choose  if  it  is  deception  or  not  and  then  tell  us  the 
correct  answer  and  explain  why. 

Get  someone  to  write  a  good  program  with  UI  amenities  that  work. 

Make  the  screen  bigger;  input  the  ability  to  go  back  a  few  seconds  in  the  video. 

Don't  go  through  things  twice 

By  breaking  down  some  of  the  examples  and  actually  showing  the  deception. 

Fine  tune  it. 

Full  screen  view 

Add  more  control  over  the  video.  Some  moved  too  fast  and  some  was  not  too  clear 
or  easy  to  understand...  I  would  have  liked  to  rewind  and  watch  again. 

Instant  notification  of  wrong  answers  and  why  they  were  incorrect 

Increase  the  size  so  it’s  easier  to  view 

Highlighted  key  words  in  text 

I  don't  know,  it  seems  like  I  already  knew  this 

Don't  know 

Use  a  better  GUI  interface.  Have  the  video  part  zoom  in  or  something  instead  of 
being  small  and  separate. 

Can't  think  of  anything. 

"Rewind"  button,  instant  feedback  on  quizzes,  use  actors  instead  of  professors  to 
read  the  transcript...  they  would  be  better  at  keeping  the  student  involved  and  interested 
More  concrete  examples  of  lying  and  such 
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Explain  a  little  better  before  jumping  into  the  next  topie  or  activity.  I  felt  like  it 
should  have  been  made  clearer  when  it  was  test  time  what  exactly  you  were  looking  for 
when  answering  the  questions.  Like  once,  I  thought  what  the  person  did  was  deceptive 
but  it  seemed  as  though  she  was  telling  the  truth. 

Refer  to  question  I's  answer. 

More  examples.  Maybe  add  a  couple  of  more  quizzes. 

A  quick  tutorial  so  that  users  know  what  they  can  and  can't  do.  For  example,  I  didn't 
touch  anything  during  the  training  in  fear  that  I'd  have  to  start  over... later  I  discovered  by 
the  questions  asked  that  this  wasn't  the  case. 

On  the  video  examples  show  the  people's  face  with  better  lighting. 

Slow  the  pace  down  and  have  a  short  review  in  the  middle  of  the  lecture 

Shorter,  less  complex  lessons.  This  would  be  a  great  system  for  those  types  of 
subjects. 

I  believe  the  program  is  a  great  tool 

I  feel,  overall,  that  it  is  a  great  learning  tool  for  those  that  can  learn  from  this  type  of 
media. 

Break  the  lecturer’s  information  down  some  more  and  include  more  verbal  and 
visual  examples  as  the  lecturers  describe  the  different  scenarios. 

More  examples  and  more  detailed  and  specific  explanations  on  the  examples. 

I  like  the  multimedia  aspect.  The  instructors  may  be  more  dynamic  if  the  video  was 
shot  with  them  teaching  in  front  of  a  group. 

Larger  screen  (video) 

One  aspect,  slides  or  lecturer,  not  both 

Have  nothing  other  than  traditional  methods  of  slides  to  compare  to.  Nothing... 

Show  more  examples  during  the  lecture  as  individual  topics  are  covered.... point  out 
aspects  of  the  scenarios  that  illustrate  the  topics  being  covered. 

Be  more  consistent...  The  text  in  the  bottom  left  hand  comer  didn't  always  match  the 
audio  in  the  video  clip.  That  made  it  somewhat  confusing  to  try  to  follow  along  with  the 
text. 

Improve  sound  quality. 

Larger  video  screen 

After  answering  a  question  I  would  like  to  see  the  answers  before  going  to  the  next 
slide 

Make  it  adaptable  for  each  individual.  I  would  cut  out  the  text  below  the  video 
screen  to  prevent  myself  from  looking  down  there.  I  could  then  concentrate  on  the  audio 
and  the  slides. 

Slides  need  to  be  more  concise  and  information  about  what  is  being  said  needs  to  be 
revealed  a  little  at  a  time  on  the  slide  (animate  the  slide  with  the  video);  feedback  on  how 
the  user  is  doing 

Answered  on  the  last  question 

More  video/audio  examples  after  definitions  and  explanations. 

Place  pauses  between  lecture  sections  to  allow  review  of  the  lecture. 

1)  After  giving  examples  of  deception  and  tmth,  list  the  indicators.  For  some 
examples,  the  instmctor  says  it  is  deceptive,  but  does  not  state  why  he  thinks  this. 

It  needs  better  examples. 
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Make  the  lesson  (slides/instruetor)  a  larger  part  of  the  sereen.  Make  the  show  require 
more  interaetion  to  keep  moving.  It's  easy  to  let  the  trainer  move  automatically  and  then 
the  student  loses  interest  and  is  less  likely  to  pay  attention. 

I  cannot  really  think  of  a  way  to  improve  because  I  don't  think  there  are  really  any 
improvements  to  be  made. 

Speed  up 

None,  it  is  effective.  I  just  think  that  sitting  in  front  of  a  computer  can  sometimes  be 
demotivating  and  distracting. 

A  rewind  option  for  the  videos  would  be  helpful  to  re -play  just  viewed  segments 

A  few  more  examples,  maybe  try  and  find  more  cues  to  look  for 

I  could  careless  about  the  history  of  the  research  of  deception,  keep  it  simple  and 
brief.  Give  me  the  tools  that  I  need  to  detect  deception 

Make  it  so  you  can  replay  the  questions 

Improve  navigation  features  to  allow  advancement  in  any  stage  of  the  application 

More  examples 

Pull  Video  for  instructors;  more  feedback  on  answers  given 

Small  Interface  cosmetics  changes  (font,  screen  size,  resolution,  etc.)  as  well  as  a  few 
more  control  options  and  menus. 

Have  not  used  it  enough  to  answer 

No  more  in  depth  on  the  "how  to  detect".  You  gave  plenty  of  info  on  the  background 
and  introduction  but  the  meat  of  the  lesson  was  not  enough 

Not  Sure.  Overall,  pretty  good. 

Video  shots  should  be  head  on.  Would  also  like  to  see  if  my  answers  were  correct  or 

not. 

If  there  were  a  way  to  take  notes  or  highlight  items  throughout  the  training.  For 
example,  some  way  to  be  able  to  bookmark  or  insert  notes  for  future  references. 

Reduce  the  amount  of  information  in  the  program.  I  realize  there's  a  lot  to  be 
delivered,  but  take  it  in  phases.  Also,  insert  an  answer  key  to  show  results  after  the 
quizzes. 

I  think  it  is  already  a  better  tool  than  I  have  seen  used  in  most  training  environments. 

Add  some  interaction  such  as  many  examples  that  individuals  can  go  though  on  their 
own.  Also  placing  more  instructions  before  things  likes  these  answers  because  I  was 
unaware  that  there  would  be  more  than  one  text  box  until  I  saw  the  person  beside  me  was 
on  question  3 

Aside  from  the  cutting  off  of  some  of  the  audio  feed,  I  think  that  it  is  an  excellent 
program.  It  is  very  user  friendly. 

Simplified  a  little  more;  a  little  slower  pace 

Split  the  lecture  into  smaller  sections.  Give  more  visual  examples  and  remove  the 
sections  that  had  no  visual  or  audio 

Cut  down  on  time  involved.  I  felt  that  some  of  it  was  repetitive. 

Addressing  the  voice  problems  I  experienced  would  help. 

Reoccurring  use... 

Get  rid  of  some  of  the  extras  on  the  viewer  and  add  a  "go  back"  button. 

Once  again  tell  me  what  the  answers  to  the  questions  are  so  that  way  I  can  look  back 
and  see  what  observations  I  missed  in  my  assessment. 
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Have  more  quizzes  during  the  lecture  versus  at  the  end.  Point  out  this  is  what 
majority  people  do  when  deceiving  but  some  have  perfected  the  "tell-tell"  clues  more 
than  others 

Have  a  pause  after  a  few  lectures,  include  a  review  period  or  some  time  where  the 
student  can  go  back  and  review  information  then  continue  where  they  left  off. 

Rather  than  using  a  single  form,  place  each  instruction  medium  in  its  own  resizable 
window  so  users  can  increase  the  relative  size  of  the  information  that  most  interests  them. 
Also,  allow  the  user  to  control  the  speed  of  slides.  Many  flipped  before  I  had  finished 
reading  them. 

Give  more  training  and  more  experience 

Provide  more  examples  of  truth/deception  for  cues 

In  my  opinion  the  word  search/ask  a  question  is  not  needed  in  a  timed  situation  but  if 
you  are  using  agent  99  in  an  untimed  situation  I  could  definitely  see  the  benefits  of  using 
the  word  search. 

Less  clutter 

Better  video  examples 

Include  more  commentary  that  points  out  the  signals  of  deception 

Not  too  much  more  than  suggestions  already  stated 

Have  the  written  script  scroll  when  the  instructor  speaks  beyond  what  is  on  the 
screen  just  to  make  it  feel  like  you  aren't  doing  too  many  things  at  once 

For  the  reading  examples/scenarios,  don't  assume  I  read  that  fast  before  going  on  to 
the  next  thing 

Nothing 

I’m  not  sure.  I  learn  best  when  I’m  interested  in  the  subject  matter. 

Slow  down  the  slides  that  I  am  supposed  to  read  or  make  so  that  I  can  change  them 
manually.  Remove  the  timer  so  that  you  can  complete  the  entire  training. 

More  memory  to  help  stop  delays. 

Maybe  more  examples 

Test  the  software  better. 

Give  more  time  to  read  the  examples. 

Maintain  the  same  volume  levels  on  all  the  video  examples,  lectures,  and  audio 
playback. 

Check  out  some  of  the  Defense  Acquisition  CBTs:  ACQ  101  or  SAM  101.  They  are 
very  interactive  and  enhance  learning  in  my  opinion.  This  CBT  was  a  little  primitive  in 
nature.  The  top  right  box  should  be  a  drop  down  menu.  The  text  on  the  bottom  left  next 
to  the  slides  on  the  bottom  right  is  kind  of  distracting  especially  when  you  can't  advance 
or  slow  down  on  your  own. 

1 .  Break  between  sections  or  prompt  to  move  on  to  next  section  or  choose  another 
section. 

Not  time  it 

Louder  audio  volume. 

Match  the  text  to  the  lecture  exactly.  Other  than  that  it  was  put  together  very  well. 

Works  fine  for  me  just  the  way  it  currently  is. 
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Allow  the  user  more  eontrol  over  the  entire  module  exeept  for  the  posttest.  I  did  not 
have  the  opportunity  to  get  through  the  entire  module  due  to  problems  with  a  pop  up 
window  overlapping  a  leeture. 

Maybe  take  away  the  ehapter  overview  part  - 1  tended  to  look  at  it  to  see  how  mueh  I 
had  left  to  view,  so  it  was  a  bit  distraeting. 

Larger  video  area  and  remove  the  narrative  from  the  sereen. 

Remove  the  time  eonstraints  whieh  will  allow  everyone  to  eomplete  the  session  and 
reeeive  the  full  benefit  of  the  information  being  presented. 

Integrate  it  with  all  of  the  Air  Foree  CBTs  as  soon  as  possible. 

Improve  videos 

More  examples  and  feedbaek  to  know  whether  you  have/have  not  deteeted  deeeption 
eorreetly  on  examples. 

5,  In  what  learning  situations  do  you  think  you  would  like  to  use  Agent99 
Trainer? 

Professional  military  development  for  new  supervisors  or  at  any  managerial  training 
Only  when  no  teaeher  is  available. 

It  might  be  good  in  plaee  of  our  eurrent  CBT  training 

Training 

None 

None. 

Not  sure 

It  would  do  a  better  job  than  CBTs 

Short  diseussions  or  training  elasses.  A  60+  minute  briefing  is  too  long  to  stare  at  a 
eomputer  sereen. 

SATE  training  and  maybe  some  of  the  COMPUSEC  and  EOAC  trainings. 

None 

It  would  be  a  better  elassroom  supplement  than  the  standard  CBT,  so  replaeing  a  few 
of  the  old  ones  with  this  program  would  be  helpful. 

As  an  option  not  to  go  TDY  for  the  training 
None 

Any  form  of  training  not  requiring  hands-on 
Don't  know 
Tree  time 

Topics  where  examples  and  interviews  need  to  be  shown  a  lot 
Online  classes  or  training  from  a  university,  replace  the  current  CBTs  that  the  AE 
uses  with  Smart  Eorce.  Perhaps  implement  more  computer-based  training  using  Agent99 
in  place  of  some  TDY  training  in  BCOT  or  ASBC. 

Distance  learning  perhaps  as  a  supplement  to  a  course— I  would  definitely  prefer  an 
actual  classroom  setting  though 
Simpler  lessons 
I  do  not  know 

If  I  could  use  Agent99  Trainer  instead  of  going  to  BCOT  for  three  months,  I  choose 
Agent99. 
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Day  to  day  operations.  Supervisor/trainee  relations. 

Online  eourses,  CBT  eourses  or  Defensive  Driving  (Traffie  Sehool) 

Military  or  edueational  (off-duty)  training. 

Situations  like  the  one  presented  here  (deeeption).  Video  examples  of  stuff  with 
immediate  analysis  of  right  &  wrong  answers 
Any 
PME 

Annual  training  sueh  as  Law  of  armed  eonflict,  different  safety  training,  etc. 

Annual  Air  Force  Training  Courses 

In  conjunction  with  class  lectures.  Sort  of  like  a  compliment  to  course  study. 

For  an  online  course.  I  think  this  is  an  excellent  way  to  conduct  distance  learning 
courses;  however,  there  should  be  an  option  to  be  able  to  actually  communicate  with  an 
instructor  for  more  specific  questions. 

I  don't  think  it  should  be  the  only  way  of  teaching.  I  thought  it  was  effective  but  as  a 
whole  I  have  difficulty  staying  focused  on  computer  based  training.  I  think  it  would  be 
best  used  as  a  supplement  or  a  reference  to  lecture. 

Here  is  a  good  idea.  In  my  current  technical  class,  have  the  class  initially  taught  in 
person.  Allow  the  student  to  use  this  system  to  refresh  or  go  over  information  again  at 
his  or  her  choosing.  Also  I  like  the  idea  that  the  system  can  be  web  enabled. 

None 

Those  where  traditional  classroom  teachings  are  not  available,  such  as  remote 
college  courses. 

Specialized  training  on  deception 

Obviously,  a  human  instructor  is  always  preferred  to  any  CBT,  but  this  program  was 
very  easy  to  use.  I  can  imagine  using  this  in  any  number  of  situations. 

1 .  Any  subject  matter  requiring  video 

It  would  be  great  to  reinforce  traditional  teaching. 

Any 

The  setup  today  work  out  great 

I  think  it  would  word  great  with  learning  new  tasks  as  long  as  the  individual  could 
get  some  sort  hands-on  while  going  through  the  training. 

Learning  from  home,  correspondence,  CBT 
Any;  this  is  much  better  than  a  CBT 

Situations  where  the  DoD  is  paying  an  instructor  to  be  static  at  the  front  of  a 
classroom  and  lecture  the  same  material. 

I  find  this  would  be  a  great  application  to  build  CBT  lectures  for  upgrade  training 
and  AFCS  specific  training  within  squadrons 
Counseling  situations 
Long  distance/correspondence 

Information  assurance  sensitivity.  AF  examples  showed  how  a  spy/thief/conman 
may  try  to  get  information  and  escape  when  someone  finds  him. 

As  a  basis  for  deception  training. 

Any  CBT  training 

I  can  think  of  a  lot  of  applications  -  some  things  that  are  already  CBT,  but  would  be 
better  if  incorporated  with  Agent99  -  such  as  the  annual  LOAC,  Anti-Terrorism,  or  ORM 
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training.  Basically,  this  is  the  best  CBT  platform  that  I  have  used  and  would  be  willing  to 
use  it  for  any  training  that  eould  be  eomputer  based. 

None,  I  prefer  leeture  from  an  instruetor/trainer. 

None 

Basie/Intro  Courses 
CBTs 

In  learning  information  within  limited  time  eonstraints.  Also,  it  would  be  useful  in 
deployed  situations  where  we  may  need  to  learn  information  in  a  safe  and  effieient 
manner. 

In  pretty  mueh  any 
Speeial  ops 

Online  eollege  eourses.  On  the  job  training. 

Online  eollege  eourse  work,  CBT  training.  Annual  safety,  suieide,  or  other  general 
briefings. 

None  really 

I  believe  it  could  be  use  for  any  training  and  learning  aetivity  that  does  not  require 
physieal  interaction  (labs,  experiments,  etc.) 

Most  CBTs  for  example,  CBTs  for  Ciseo  ete 
Reoeeurring  training 
Opsee,  compusec,  and  eomsee  training 
LOAC 

CBT  at  my  base  or  you  eome  TDY  and  do  the  elass 
On-line 

One  on  one  and  voiee  reeognition  training 
1)  Refresher  training 

I  would  like  to  see  it  in  New  Comer  Orientation  for  bases.  I  think  it  would  be 
beneficial  to  employees. 

Tm  planning  on  getting  my  Master's  Degree  via  on-line  learning  and  I  imagine 
eoursework  taught  through  Agent99  would  be  easier  to  understand  and  lead  to  better 
results  than  other  types  of  media. 

On  the  job  training. 

Home  base,  not  at  BOOT 

Anyone  ean  legitimately  benefit  from  the  program. 

For  quiek,  one-time  eourses  that  would  normally  take  only  a  day  or  so.  Tm  not  sure  I 
would  prefer  this  method  of  learning  for  longer-term  courses. 

Initial  and  annual  training. 

Tm  really  not  sure.  The  only  thing  that  really  eomes  to  mind  is  to  ensure  proper 
training,  espeeially  when  dealing  with  OPSEC,  COMSEC,  and  COMPUSEC. 

Probably  in  a  eustomer  service  situation  (the  day  to  day  interaetion) 

Short  CBT  in  my  job  would  aetually  be  a  good  plaee  for  these.  Espeeially  when  the 
student  would  benefit  from  examples... 

In  eonjunetion  with  a  elass 
CBT 

Air  Eoree  training  requirements.  Although,  it  should  only  be  used  as  a  tool  along 
with  leeture. 
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I  think  it  would  be  incredible  to  use  agent  99  for  ROTC  cadets  just  before  the 
commission  to  use  this  and  give  them  a  complete  breakdown  on  how  to  in  process  at  their 
base  and  also  give  them  a  good  understanding  of  the  base  they  are  reporting  to.  On  active 
duty,  probably  for  financial  counseling  for  those  in  need  of  it. 

None 

Interactive  lecture 

Prefer  face-to-face  interaction. 

Language 

Learning  new  languages 

As  an  introduction  to  lA  and  security  for  all  personnel  (i.e.  all  personnel  at  AFSOC 
should  take  this  CBT  online) 

Subject  material  that  takes  a  lot  of  examples  to  understand  it 
Basic  or  101  course. 

Not  sure.  Perhaps  step  by  step  computer  work  using  examples. 

At  my  desk  at  work 
All 

Similar  applications  to  the  deception  training. 

CBTs 

Upgrade  training 
To  replace  the  current  CBTs 
Annual  training 

Pretty  much  any  recurring  training. 

In  an  at  home-based  environment  where  I  can  take  my  time  with  it,  stop  and  replay 
info  as  needed 
Tm  not  sure. 

One  could  put  the  entire  BCOT  course  in  an  Agent  99  trainer  and  save  millions  of 

$$'s 

For  new  personnel,  say  basic  trainees  or  tech  school  students.  Coming  from  an 
Intelligence  background  and  working  with  classified  material  this  trainer  would  make  life 
much  easier  on  trainers  out  in  the  field.  Getting  a  trainer  like  this  to  train  personnel  would 
strengthen  security  awareness  and  help  teach  some  valuable  skills  need  in  the  field. 

I  think  it  could  be  used  for  a  number  of  various  programs  job  specific  as  well  as 
example  based,  because  it  allows  the  person  to  review  items  as  well  as  focus  examples.  I 
can't  think  of  any  specific  type  of  area  where  you  could  not  tailor  this  type  of  program  to 
deliver  the  necessary  training. 

For  short  subjects,  periodic  training  on  specific  topics. 

This  is  good  for  all  situations  I  can  think  of. 

Agent99  trainer  would  be  good  in  any  academic  setting. 

Any  computer  based  training  for  computers,  on-line  classes,  correspondence  courses, 
etc...  it  would  be  very  useful  in  all  of  them. 

Like  "FYI"  info,  I  believe  important  topics  should  be  emphasized  with  a  CBT  and 
then  discussed  in  person  (group  environment  preferably) 

In  pretty  much  any  CBT  course. 

Computer  training,  network  information.  Information  assurance.  Security  awareness 
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It  would  be  useful  for  people  who  are  responsible  for  protecting  information  and 
resources  (security,  SCIF,  controlled  area  workers).  I  do  not  think  it  would  be 
appropriate  to  disseminate  the  information  widely  to  the  general  population  of  the 
military. 

Initial  and  recurrent  deception  training  (similar  to  AF  Info  Awareness  training). 

6,  Other  comments: 

If  additional  features  were  built  in,  it  could  prove  a  very  useful  additional  training 

tool. 

Have  the  pop-up  screen  automatically  take  advantage  of  the  entire  screen  size 
This  was  pretty  fun  and  very  interesting...  great  job! 

I  think  that  this  is  a  good  idea  to  teach  people  how  to  judge  others. 

Overall  felt  that  this  is  a  great  program.  Having  been  a  sluggish  day  for  me  today,  I 
felt  that  it  just  added  more  to  the  feeling  of  being  tired. 

I  think  Agent99  Trainer  is  an  excellent  tool  that  is  easy  to  use  and  conducive  to 
learning,  especially  in  an  outside-the-classroom  type  course.  I  also  found  it  quite 
enjoyable. 

Good  job  on  the  system  I  really  liked  it. 

While  taking  the  test,  could  we  look  up  the  answers?  Based  on  the  usability  test,  it 
appeared  so.  However,  I  don't  remember  that  being  explained. 

The  program  itself  was  extremely  user  friendly  and  simple  to  use,  but  the  monotone 
video  was  somewhat  boring.  Also,  it  would  be  helpful  to  have  more  explanation  of  the 
example  video  clips...  Not  having  any  prior  experience  with  this,  it  would  have  been 
useful  to  have  an  explanation  of  what  I  had  just  seen  after  some  of  the  clips  to  point  out 
some  of  the  things  I  was  supposed  to  be  looking  for.  Overall,  this  was  an  excellent 
program. 

Maybe  slow  down  the  topics  a  little  more  in  case  people  want  to  take  notes 
Is  this  for  learning  deception  or  are  you  trying  to  get  feedback  on  Agent99? 

Nice  try 

Good  job!  Looks  like  you  did  a  lot  of  work  in  making  this. 

Be  careful  not  to  have  TOO  much  info,  as  in  length  - 1  was  starting  to  zone  out  near 
the  end  because  of  the  constant  flow  of  information 

It  would  be  nice  to  be  able  to  change  where  the  video  is  playing;  if  you  missed 
something  then  be  able  to  back  up  the  clip  a  couple  seconds 
give  more  time  to  learn  and  understand 
Pop-up  screen  very  irritating 
Great  product. 

Well  put  together  software  with  only  a  few  glitches.  I  can  admire  the  work  put  into 
the  program  itself;  however  I  feel  the  content  needs  more  development 

Overall  this  training  tool  was  very  user  friendly  and  easy  to  navigate.  I  hope  to  see 
this  in  future  Air  Force  technical  training  school  courses.  Maybe  even  one  day  be 
allowed  to  attend  Air  Force  technical  training  from  home.  By  doing  so  it  would  allow 
families  to  stay  together  and  not  have  to  be  separated  as  much.  If  it  was  to  be  used  to 
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train  in  this  way,  the  test  eould  be  eonducted  at  the  loeal  bases  and  e-mailed,  faxed,  or 
mailed  to  the  appropriate  sehool  for  grading. 

During  the  videos,  the  Lt  offered  too  mueh  information  to  an  individual  who  knew 
nothing,  i.e...  names  of  meetings  and  things  of  that  nature.  The  information  he  provided 
alone  was  enough  for  the  intruder  to  possibly  use  on  another  member  at  a  different 
loeation. 

Good  program  overall 

This  would  be  good  for  supervisors  and  managers  to  take.  It  would  help  them 
identify  dishonest  subordinates. 

Overall,  the  information  was  good.  I  enjoyed  taking  the  test  and  it  gave  me  some 
tools  to  help  better  identify  deeeption 

My  only  other  eomment  is  that  if  you  use  examples  that  require  the  user  to  read  the 
example  set  it  up  so  that  when  they  are  done  reading  the  page  that  they  ean  eliek  a 
"continue  button"  and  read  the  next  slide  when  they  are  ready.  People  don't  read  at  the 
same  speed  and  I  missed  a  couple  of  the  lines  a  few  times. 

Excellent  program  overall 

Overall  good  experience  using  the  trainer. 

Before  implementing  any  CBTs  in  this  format,  look  at  what  the  Defense  Acquisition 
University  has  in  place  for  their  computer  based  classes.  They  have  some  great  tools 
available  for  students. 

Good  content  and  a  lot  of  information. 

I  thought  it  was  very  interesting.  Where  can  I  purchase  or  retake  this  course  without 
having  to  rush  or  being  timed? 

It  was  an  interesting  experience. 

All  in  all  very  good  training,  the  environment  leaves  a  bit  to  be  desired.  My  lingers 
are  frozen  and  I  had  a  difficult  time  concentrating  during  the  final  portions 

The  last  thing  I  would  have  Irked  is  to  have  had  was  a  better  idea  about  how  long 
sections  were  and/or  the  number  of  questions  I  would  be  expecting  to  answer  for  timing 
purposes. 

Great  idea,  program  interface  looks  dated,  but  once  polished  should  be  a  great  tool, 
(a  standardized  interface  would  be  smart;  the  AF  has  too  many  systems  that  do  the  same 
thing).  Thanks. 

With  reference  to  the  deception  course: 

Great  Job. ..Good  Luck 

Nice  simulator-I  feel  it  has  real  value  for  deception  detection  efforts;  especially  the 
"real-world"  scenarios  you  are  likely  to  encounter  in  anNCC  with  contractors/support 
personnel  on  a  daily  basis. 
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